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THE FEDERAL PROBATION and pretrial 
services system has been working diligently 
to implement organizational and process 
changes to improve our outcomes. We feel it is 
a good time to share what we've learned. Some 
of the subjects covered in this issue simply 
describe who we are as a system of officers 
and clients. Other articles discuss what we are 
trying to do. Still others spell out what we've 
learned so far. While we have used this journal 
many times as a venue to describe our work, 
we have begun in recent years to dedicate 
entire issues to providing an update on where 
we stand and how we are progressing. This 
is important now, as we are more purpose- 
fully identifying evidence-based principles 
and very consciously basing our decisions on 
the best evidence available. This is of course, 
a long process, and we plan to share what we 
learn as we go, in hopes that the field of com- 
munity corrections can keep moving the ball 
forward, and that our system can contribute 
to that forward motion. 

By accepting the challenge to use the best 
available evidence to make decisions, we 
inherently accept that we must be participants 
in the gathering of that evidence. This is an 
exciting new responsibility for a group of pro- 
bation and pretrial services officers. We are 
accepting the call not simply to adopt what 
has worked for others, not simply to continue 
what we have done for years, not simply to fol- 
low our guts on what we think we know works, 
but to regularly evaluate and reevaluate. Most 
important, we accept the responsibility to alter 
our practices when the evidence shows there 
is a better way. While all of this is difficult 
work, it is made more challenging in a time 


of budget constraints: more challenging—and 
yet more important. As we work to maintain 
and improve our outcomes, we do so with 
fewer resources. Now more than ever we 
must be willing to replace less effective pro- 
cesses with more effective ones, and focus our 
resources on those on whom we will have the 
greatest impact. We have some very encourag- 
ing news to present within this journal. We 
are at the beginning and we know that Rome 
wasnt built in a day. But we believe we have 
the architects, the builders, and the bricks to 
eventually build a Rome. 

The lead article by John M. Hughes takes 
a step back and reviews the thinking going on 
in community corrections 50 years ago. It may 
be surprising to find authors writing in 1961 
expressing thoughts very similar to those we 
are expressing today in this and many other 
criminal justice publications—emphasizing 
the officer using his authority wisely to sup- 
port the behavior change of clients. Hughes 
goes on to comment on another writer's belief 
that our failure to measure outcomes and 
evaluate our practices has been our largest 
stumbling block, but then suggests that that 
block has largely been removed. 

If we are to evaluate fully, we must look 
carefully at all of the pieces of the model. An 
important piece is that of the officer, for our 
assumption is that he can influence behavior 
change in the client. What characteristics are 
we looking for in our officers, and do we have 
them in our current staff? The next two articles 
begin to describe what federal probation offi- 
cers look like. Whetzel et al. present the results 
of a survey of officers’ attitudes. The find- 
ings from three districts indicate that officers 


Nancy Beatty Gregoire, Guest Editor 
Chief, Program Administration Division 
Office of Probation and Pretrial Services 
Administrative Office of the U.S. Courts 


are well-balanced in their approach, and also 
reveal some differences in attitude depending 
upon role within the office. Though admittedly 
based upon a small sample, these are some very 
interesting findings. Whetzel and Lowenkamp 
then attempt to answer the same question 
from the perspective of those whom we hope 
to affect. They describe the results of a survey 
of offenders. Both of these surveys represent 
small numbers within our system, but both are 
encouraging in that they suggest we may, in 
fact, have the right people for the job. 

The next two articles describe the newest 
pieces of our system's method of determining 
who is before us—the risk assessment instru- 
ments. Johnson et al. provide the background 
for the development of the Post Conviction 
Risk Assessment, or PCRA. For reasons delin- 
eated in the article, the federal system decided 
to develop its own fourth-generation assess- 
ment instrument, rather than purchasing an 
off-the-shelf product. Johnson et al. describe 
the dataset used for the construction and 
validation process for the PCRAs development. 
Cadigan and Lowenkamp next discuss the 
implementation of the assessment instrument 
developed to assist in pretrial decision making, 
the Pretrial Risk Assessment (PTRA). While 
Lowenkamp and Whetzel (2009) described the 
development and introduced the instrument 
itself, this article discusses the system's prog- 
ress in implementation, and more important, 
begins to evaluate the impact of the PTRA on 
release recommendations and release rates of 
defendants in a couple of pilot districts. 

The series of articles that follow set out 
what we are trying to do and provide a snap- 
shot in time of what in fact is happening in 
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various areas. Hurtig and Lenart, as deputy 
chief probation officers in the federal system, 
describe their leadership of a working group 
formed to advise the Administrative Office 
of the U.S. Courts on improving supervi- 
sion outcomes. They delineate the working 
groups role in laying out the “Evidence-Based 
Practices Blue Print: Incorporating EBP in the 
US. Probation and Pretrial Services System,” 
along with the content of the “Blue Print,’ 
which calls for three important products: an 
education piece for all of the system, a national 
risk/needs assessment instrument, and a 
training program for officers on the use of 
hands-on supervision skills aimed at reducing 
rearrest. Then comes Meierhoefer’s summary 
of the results of a survey of federal probation 
chiefs asked about their judge-involved super- 
vision programs. This survey is part of a larger 
effort by the Federal Judicial Center, at the 
request of the Judicial Conference Committee 
on Criminal Law, to study the operations 
and cost-effectiveness of judge-involved reen- 
try programs that have been implemented 
in recent years and modeled on state and 
local drug and reentry courts. Meierhoefer 
describes the variety in program philosophies 
and practices and discusses the need for further 
evaluation of outcomes and cost-effectiveness. 
In “Applying Implementation Research to 
Improve Community Corrections,’ Alexander 
reminds us that a truly evidence-based system 
would rely on research to direct the manner in 
which new ideas or processes are introduced. 
Many practitioners are not aware that there is 


a body of research on best ways to effectively 
implement change within an organization. 
The final five articles in this issue share 
what we have learned from early evaluations on 
our work. Four of them represent recent find- 
ings within the federal system, while the fifth 
is a meta-analysis based on pretrial research 
outside of the federal system. Oleson et al. and 
Robinson et al. discuss some findings related to 
the two largest products called for in the “Blue 
Print,’ the post-conviction risk assessment 
instrument, and the training on supervision 
skills, or “Strategic Skills Aimed at Reducing 
Rearrest” (STARR). In “Training to See Risk: 
Measuring the Accuracy of Clinical and 
Actuarial Risk Assessments Among Federal 
Probation Officers,’ Oleson et al. test whether 
the use of the new post-conviction risk assess- 
ment instrument (PCRA) improves the ability 
of officers to accurately assess risk. While a 
large body of research suggests the dominance 
of actuarial risk instruments over officers’ 
professional judgment, there was an interest in 
testing that suggestion within the federal sys- 
tem. The news is good: the use of the PCRA led 
to more accurate and consistent assessments. 
Robinson et al. present a randomized study 
with a pre-post test design that looks at whether 
training officers in the use of core correctional 
supervision skills (a program with the acro- 
nym STARR in the federal system) will result 
in the increased use of the skills, and whether 
the increased use of the skills would improve 
rearrest rates of the clients under supervision. 
Robinson et al. found that the training both 


increased the officers’ use of the skills and 
was associated with lower rearrest rates. These 
findings are very encouraging and support the 
continued training and roll-out of the STARR 
program as we continue to evaluate and adapt 
to new information. 

As the Meierhoefer article indicates, many 
federal courts have initiated some version of a 
reentry or drug court or other judge-involved 
supervision program in the past couple of 
years. Several of those programs have sub- 
jected themselves to study or evaluation. 
Vance provides a history of the drug and reen- 
try court movement and gives an overview of 
the federal programs. He then summarizes 
evaluations of three such federal programs. 

Next comes “Preentry: The Key to Long- 
Term Criminal Justice Success?” Cadigan and 
Lowenkamp analyze the relationship between 
success of a defendant on federal pretrial super- 
vision and the success of that same person on 
post-conviction supervision. This is the first 
study to demonstrate such a correlation within 
the federal system, and perhaps the first pub- 
lished study showing this relationship. 

Finally, “Identifying the Predictors of 
Pretrial Failure: A Meta-Analysis,’ by Bechtel, 
Lowenkamp, and Holsinger, examines indi- 
vidual risk factors in pretrial assessments to 
determine which factors are indeed associated 
with pretrial failures. Some surprising findings 
may support the review of current risk assess- 
ment instruments for validity and reliability. 
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Assistant Director, Office of Probation and Pretrial Services 
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WHAT WERE CONTRIBUTORS to Federal 
Probation writing about 50 years ago, in 
September 1961? That's the year that President 
Eisenhower warned of the “military industrial 
complex” in his farewell address, and NASA‘ 
Project Mercury made Alan Shepherd the first 
American in space. John F. Kennedy estab- 
lished the Peace Corps, Roger Maris broke 
Babe Ruth’s home run record, and Joseph 
Heller published Catch-22.' It was an interest- 
ing year for sure, but what was going on in 
community corrections, as glimpsed in the 
September 1961 issue of Federal Probation?’ 
John J. Galvin, Warden of the Federal 
Reformatory in El Reno, Oklahoma, wrote 
about the need for a common philosophy of 
treatment that would bring together pun- 
ishment, humanitarianism, and science. He 
explained that punishment alone may not be 
enough to change behavior. Likewise, a soft 
approach (he called it “an uncritical accep- 
tance of the old dictum: “There but for the 
grace of God go I”) by itself may inadvertently 
convey to the offender that he does not need 
to change, that his behavior is understand- 
able given his background and circumstances. 
Galvin saw the need to stop “vacillating” 
between the two general approaches, and 


' Internet: infoplease.com; factmonster.com 


* Victor H. Evjen, editor, and William A. Maio, Jr. 
comprised the editorial staff. The journal was pub- 
lished by the Administrative Office of the United 
States Courts, as it is today, but in 1961 it was pub- 
lished in cooperation with the Bureau of Prisons. 
Warren Olney Iil was AO Director and Robert 
F. Kennedy was Attorney General. The Bureau of 
Prisons Director was James V. Bennett and the AO’s 
Chief of Probation was Louis J. Sharp. 


called for a “lasting synthesis of ideas” that 
employs science and would allow us “to 
choose differently at different times between 
the two approaches, depending upon the cir- 
cumstances of each situation.”* 

Arthur E. Fink, Dean of the School of Social 
Work at the University of North Carolina, 
discussed the need for offenders to come 
to terms with authority and limits and take 
responsibility for themselves. He discussed 
the futility of lecturing an offender who is not 
ready to change. Simply telling an offender 
to behave himself or admonishing him for 
unwanted behavior brought this observation 
by Fink: “I am moved to observe that such 
an approach if not downright harmful is of 
limited usefulness or of no use at all, because 
it is based upon a misleading notion of human 
behavior.” He pointed out that to be effective 
we must “engage the offender in the process of 
doing something about himself” We need to 
help the offender see the need to change and 
provide him opportunities to make it happen. 
Fink notes that the officer's approach to work- 
ing with offenders: 

...Will depend to a great extent upon 
his convictions about people—his 
respect for them as human beings, 
with all of their shortcomings; his 
appreciation of the uniqueness of 
each person with whom he is work- 
ing; his belief in the capacity of 
people to change; and his convic- 
tion that true change must come 
from within. As he works on these 


premises he can approach each of 
his parolees as individuals who have 
difficulties of a serious nature and 
who need help in getting themselves 
straightened out—and that he has 
the skill to help.* 

Serapio R. Zalba, Executive Director, 
Northern California Services League of San 
Mateo County, discussed how an individual's 
behavior is related to physical, psychological, 
and social needs. Most of us satisfy our needs in 
socially acceptable, conventional ways. Others- 
those who end up in the correctional system-go 
about it in ways that are unacceptable to society. 
Zalba makes the classic argument that “(t)he 
justification for treatment in corrections is the 
belief that delinquent actions are expressions 
of underlying emotional and social problems 
and needs.” For correctional treatment to be 
effective, the person providing it must adhere 
to four concepts: 

1. There must be a genuine desire 
to understand what the offender 
thinks, does, and feels from the 
offender's point of view. 

2. Only the offender can live his 
life, and therefore he must assume 
responsibility for his own actions 
and decisions within the boundaries 
set by society. 

3. People have the capacity to learn 
and to make constructive changes. 
4. Correctional workers must use 
their authority wisely, avoiding 
excessive control over individuals 


> John J. Galvin, Federal Probation, “The Task of 
Corrections.’ September 1961. 


* Arthur E. Fink, Federal Probation, “Authority in 
the Correctional Process.” September 1961. 
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who don't need it...one approach 
applied to all offenders cannot be 
equally effective in each of the vary- 
ing situations the worker must deal 
with.” 

Wait a second. Aren't the three authors 
from 50 years ago saying the same sort of 
things that we're saying now? We could almost 
get away with changing the dates and repub- 
lishing these articles today. We were pretty 
smart back then. From this one glimpse at the 
content of Federal Probation 50 years ago, it 
appears we were on the right track. What hap- 
pened? Why did we stop? 

The obvious answer is that we were dis- 
tracted by the era of “nothing works” and the 
“tough on crime” generation of federal and 
state laws that filled the nation’s prisons and 
caused hundreds more to be built. Others 
have done a fine job of describing the cor- 
rosive impact of these two phenomena on 
community corrections, and so I need not 
repeat the history here.® It may be useful, 
however, to mention another reason cited 
in the December 1985 edition of Federal 
Probation by Harold B. Wooten, who made 
the following observation about the federal 
probation system: 

(W)e have failed miserably to criti- 
cally examine how well we have 
done, or can do, in helping offend- 
ers change their self-defeating 
behavior, and by our failure to 
discuss this issue, we have been less 
than candid. 

Wooten described the federal system's 
failure to measure outcomes and evaluate 


supervision programs, and asserted that the . 


system never really made behavior change 
a desired goal of supervision. Instead, the 
probation system was satisfied with “cursory 
supervision,’ and put almost all its attention 
on presentence investigations. He noted that 
new probation officers came into the system 
“to work with people,” but their talents were 
diverted to presentence investigations and 
surveillance-type supervision.’ 

Having attended many federal judiciary 
conferences and ceremonies in my career, Id 
have to agree with Wooten—up to a point. 


° Serapio R. Zalba, Federal Probation, “A Look at 
Correctional Treatment.” September 1961. 


° See Cullen and Gilbert, 1982, Reaffirming 
Rehabilitation for an historical review of punish- 
ment, the progressive era, and the demise of 
rehabilitation during the late 20th Century. 


? Harold B. Wooten, Federal Probation, “It's OK, 
Supervision Enthusiasts: You Can Come Home 
Now!” December 1985. 


Many times over the years, I’ve heard federal 
district judges speak to audiences of proba- 
tion and pretrial services officers and thank 
them for their excellent work. Almost invari- 
ably, they mentioned presentence reports and 
thanked the officers for their contributions 
in the difficult business of sentencing. The 
judges were sincere and meant well, but it 
always seemed a little uncomfortable for the 
officers in the room who did not prepare 
presentence reports, and instead supervised 
offenders or provided pretrial services. It was 
also understandable. 

Presentence reports have long been highly 
visible and critical to the district court docket. 
In contrast, district judges see pretrial services 
reports infrequently, and usually only are 
involved in post-conviction supervision when 
there is an apparent violation. Supervision 
successes are typically invisible. It is not sur- 
prising, therefore, that presentence reports 
would jump to mind when a district judge 
spoke to audiences of probation and pretrial 
services officers. _ 

Nowadays, however, it would not be all 
that surprising for a federal district judge to 
talk about his or her very own offender reen- 
try program. In the last three years, about 40 
districts have set up offender supervision pro- 
grams that involve judges. While the programs 
vary significantly from district to district, 
they all attempt to leverage the authority of 
the court to improve the chances of offender 
success. All, therefore, operate under the 
assumption that people can change and that 
there are identifiable ways to help them do it. 

Clearly, the sad “nothing works” era is 
over. The “tough on crime” voices no longer 
dominate the conversation. We are again 
free to explore better ways to improve the 
likelihood of offender success and protect the 
community. The excitement over prisoner 
reentry is palpable, but will it fade away as did 
the ideas from 50 years ago? I don't think so, 
and here's why. 

There is now broad acceptance in the 
federal system and the field of community 
corrections generally of the need to measure 
and study outcomes to determine program 
effectiveness. We have realized that counting 
activities and outputs and adding up expendi- 
tures tells us only that we did something and 
it cost something, not that we accomplished 
something. In the federal system, we clearly 
defined seven years ago what we want to 
accomplish with persons under supervision. 
The desired outcomes of supervision are: 


... the execution of the sentence and 
the protection of the community by 
reducing the risk and recurrence 
of crime and maximizing offender 
success during the period of super- 
vision and beyond. The goal in all 
cases is the successful completion 
of the term of supervision, during 
which the offender commits no new 
crimes; is held accountable for vic- 
tim, family, community, and other 
court-imposed responsibilities; 
and prepares for continued success 
through improvements in his or her 
conduct and condition. “Continued 
success” within the context of the 
criminal justice mission is refraining 
from further crime. Offenders “pre- 
pare for continued success” during 
the term of supervision by actively 
dealing with those circumstances 
that relate to future criminality.* 

It is now clear that officers are expected to 
help offenders change their behavior not just 
while we're watching, but for the longer term- 
after the period of supervision is over. If we 
are successful, the offender will be less likely to 
commit new crime and victimize others, and 
will be more likely to support himself and his 
family in a lawful manner. The individual and 
society wiil be better off. 

Of course, there will be recalcitrant offend- 
ers who will not cooperate, and will not 
change their behavior seemingly no matter 
what interventions are made. Our strategy in 
such cases is to respond to noncompliance 
with the condition of supervision, simulta- 
neously providing a negative consequence 
designed to deter further noncompliance and 
intervening to change the circumstances that 
led to the behavior. Officers may emphasize 
controlling strategies to detect new criminal 
behavior, including community observation’ 


* Guide to Judiciary Policy, Volume 8, Part E, 
“The Supervision of Federal Offenders” (formerly 
Monograph 109). J-Net (federal judiciary intranet), 
§ 150(a)(b) (last revised 12-10-10). 


° Community observation is fieldwork that does 
not involve a direct contact with the offender or 
collateral sources. It may be the preferred way to 
unobtrusively monitor compliance with specific 
conditions in a way that does not intrude on the 
activity itself. For example, an officer might drive 
by an Alcoholics Anonymous meeting to see if the 
offender's car is parked there or go to an offender's 
work site or residence during the start or end time 
of his or her reported work schedule, which may 
be appropriate if the offender is suspected of falsely 
reporting employment. Guide to Judiciary Policy, 
Volume 8, Part E, §450.30, op.cit. 
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and searches."® The original arresting agency, 
federal task forces, local intelligence, or com- 
munity policing meetings are good sources 
of information about an offender's pattern 
of criminal activities and associates and can 
provide valuable assistance to the officer 
in monitoring the offender's activities while 
under supervision."' 

Generally, however, we have renewed the 
assumption that people can change under 
the right circumstances, and we are pursu- 
ing-simultaneously-a two-pronged strategy 
to increase success rates and reduce recidi- 
vism. One prong is to foster the use of 
evidence-based practices (EBP) by probation 
and pretrial services officers. We are imple- 
menting risk assessment tools'? and providing 
officers with core correctional skills training’ 
proven to make interactions with offenders 
more effective. Our pursuit of EBP is well 
described in this issue of Federal Probation, so 
I will comment here only on the second prong 
of our strategic approach." 


'© The Judicial Conference Committee on Criminal 
Law set forth guidance for search and seizure 
in Search and Seizure Guidelines for Probation 
Officers in the Supervision of Offenders on 
Supervised Release, which was approved by the 
Judicial Conference at its September 2010 session 
(JCUS-SEP 2010). 


'! Guide to Judiciary Policy, op. cit. 
"2 See Johnson et al. in this edition of Federal 
Probation, and Lowenkamp and Whetzel, 2010. 


'S See Robinson et al. in this edition of Federal 


Probation. 


'* See Hughes, Federal Probation, September 2008, 
“Results-based Management in Federal Probation 
and Pretrial Services”; and Baber, Federal Probation, 
December 2010, “Results-based Framework for 
Post-conviction Supervision Recidivism Analysis.” 


We are very close to completing the tech- 
nical infrastructure by which to implement 
a results-based management and decision- 
making framework for the federal probation 
and pretrial services system. We now have the 
ability to collect records from the electronic 
files of thousands of probation officers in 
all 94 federal districts in the Probation and 
Pretrial Services Automated Case Tracking 
System (PACTS), and store these records in 
a single data warehouse called the National 
PACTS Reporting (NPR) System. 

The crowning jewel of the technical infra- 
structure is the Decision Support System 
(DSS), which draws data from NPR and 
combines it with data from other judi- 
ciary systems, the United States Sentencing 
Commission, the FBI, the Federal Bureau of 
Prisons, and the Bureau of the Census. We 
can now begin to test underlying assumptions 
about the relationship between supervision 
practices and supervision outcomes, i.e., 
between what the system does versus what it 
accomplishes, and what other factors may be 
influencing the relationship. 

In addition to putting data and analysis 
in the hands of decision makers and greatly 
enhancing our ability to conduct research, 
the technical infrastructure we have created 
enables managers at all levels of the probation 
and pretrial services system to access data for 
straightforward operational business. Having 
standard reports and dashboards only a few 
clicks away can help with monitoring inves- 
tigations, caseload sizes, location monitoring 
equipment inventory, Risk Prediction Index 
(RPI) and criminal history score trends, treat- 
ment services costs, etc. 


If the people from 1961 cited in the begin- 
ning of this article were to come back to us 
today, what could we tell them? The news 
for President Eisenhower, Alan Shepard, and 
Roger Maris wouldn't make them happy. The 
military industrial complex may be even 
bigger and more powerful than President 
Eisenhower could have imagined. NASA just 
mothballed our manned space flight pro- 
gram with the retirement of the space shuttle, 
and the home run record is now owned by 
a convicted felon whose performance was 
enhanced by more than just Wheaties. 

For the contributors of Federal Probation 
the news would be much better. We have 
embraced the balanced approach suggested 
by Warden John Galvin, adopted Dean 
Arthur Fink’s focus on increasing the motiva- 
tion of persons under supervision to improve 
themselves, and incorporated the wise use of 
authority advocated by Executive Director 
Serapio Zalba. The big difference 50 years 
later is that we now have scientific evidence 
to support the ideas promoted back then. 
They may have been right 50 years ago, but 
there was no proof. In the federal system, we 
have also built the technical infrastructure 
that will enable self-study and the strate- 
gic use of resources envisioned by Harold 
Wooten in 1985. 

I am confident that our talented cadre 
of community corrections professionals and 
our commitment to rely on rigorous empiri- 
cal research—by others and by ourselves—to 
inform policies and practices will continu- 
ously improve system performance and keep 
us on the right track for the next 50 years. 
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MUCH HAS BEEN LEARNED over the past 
three decades about how individual offender 
rehabilitation—“what works’—relates to 
reducing recidivism (Andrews, et al., 1990; 
Palmer, 1995; Gendreau, 1996; Gendreau, 
French, and Taylor, 2002; Andrews and Bonta, 
2010). Far less has been learned about the 
impact of the professional orientation of pro- 
bation officers (POs) on offender recidivism, 
although the significance of the professional 
orientation of probation and parole officers 
has been recognized by scholars and practi- 
tioners for well over 50 years (Ohlin, Piven 
and Pappenfort, 1956; Glaser, 1969; Fitzharris, 
1979; Paparozzi and Gendreau, 2005). 

In the context of probation supervision, 


applying evidence-based principles for effec- 


tive intervention is at all times colored by 
the professional orientation of POs. If, for 
example, POs favor law enforcement func- 
tions, a tepid approach toward evidence-based 
policies, programs, and practices that support 
offender rehabilitation may result. Many in 
the field of corrections would agree that staff 
orientation can “make or break” a program 
or policy (Takagi, 1973; Gendreau, Goggin, 
and Smith, 1999; Paparozzi and DeMichele, 


' Probation Administrator, Administrative Office 
of the U.S. Courts, Office of Probation and Pretrial 
Services. 
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2008; Paparozzi and Schlager, 2009). Given 
this distinct possibility, it is essential for pro- 
bation organizations to engage in rigorous 
self-analysis to determine how and to what 
extent the values and professional orientation 
of POs might interact with efforts to reduce 
offender recidivism. 

A primary function of POs is to spur 
and nurture the pro-social reintegration of 
offenders. In so doing, two essential public 
safety functions of probation are fulfilled: a) 
short-term risk management of criminals con- 
ditionally supervised in the community; and b) 
long-term behavioral reform for the purpose 
of individual offender recidivism reduction 
(Paparozzi and Hinzman, 2005). In the con- 
text of probation supervision, these public 
safety outcomes are generally accomplished 
through case management strategies involving 
some combination of law enforcement and 
social/casework activities. The importance 
of achieving a balance between law enforce- 
ment and social/casework activities and in 
achieving a proper temporal ordering of these 
activities in the context of probation supervi- 
sion has yet to be addressed by the probation 
profession (Paparozzi and DeMichele, 2008). 
A law enforcement/social worker dichotomy 
has traditionally been invoked to describe 
officers’ professional orientations, with many 
staff at all organizational levels being polarized 
at one extreme or the other. It is well-known 
among probation insiders that this dichotomy 
is a fact of professional life, and that empha- 
sis on one or the other ebbs and flows with 
the shifting political and ideological cultures 
unique to a particular epoch.* 
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Research findings suggest that the “incom- 
patibility of the control and assistance tasks,” 
results in a “chasm” that impedes effective pro- 
bation supervision (Fulton et al., 1997). Some 
probation professionals contend that this role 
conflict be resolved by abandoning the social 
work/assistance role entirely (Barkdull 1976). 
Others simply seem to accept that both orien- 
tations have a place in community corrections. 

Researchers have suggested that PO role 
conflict negatively affects organizational effec- 
tiveness and service delivery to offenders. 
Blumberg (1974) noted that unresolved law 
enforcement and assistance roles contribute 
to an uncertain professional status and “civil 
service malaise.” Whitehead and Lindquist 
(1985), in a study of probation officer burnout, 
found that 63 percent of respondents noted 
the role conflict between law enforcement 
and social casework functions and the general 
impersonal treatment of offenders as contrib- 
uting to burnout. 

Researchers and practitioners spend sig- 
nificant time and effort trying to understand 
how to reform offender behavior in correc- 
tional settings. Virtually no effort, however, 
is expended on the relationship between 
professional orientation of probation offi- 
cers and recidivism. The significance of the 
failure to examine the relationship between 
PO orientation and the success or failure of 
probationers cannot be overstated (Studt, 
1973; Paparozzi and DeMichele, 2008). The 
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very foundation upon which the delivery 
of appropriate treatment services is based 
is flawed if the individuals operationalizing 
the delivery of services are inappropriate role 
models, inflexible in their response to proba- 
tioner relapses, uncaring, cynical about their 
employing agency and the clients served, 
and the like (see Offender Survey article, this 
issue of Federal Probation). 

Even the best evidence-based principles 
must be “washed through” the filters of staff 
values and professional orientation, and it is 
logical to assume that this filtering process 
may sometimes moderate or even negate the 
positive effects of evidence-based programs, 
policies, and practices that target offender 
rehabilitation. Frontline probation staff 
socially construct the realities of probation 
supervision in the context of their every- 
day perceptions and professional orientations 
towards their job and the offenders under 
supervision. It is this everyday construction 
of reality that determines the therapeutic 
integrity of the application of evidence-based 
principles. It is at least plausible that the 
efficacy of an evidence-based rehabilitation 
program or service may be diminished by 
probation officers who do not value offender 
rehabilitation and therefore implement such 
programs less than enthusiastically. 

Examining, understanding, and mod- 
ifying, when appropriate, the professional 
orientations and attitudes of federal probation 
officers is a critical step in the adoption of 
evidence-based practices. After a brief review 
of the relevant research literature, this study 
presents preliminary findings from two sur- 
veys of staff job orientation administered to 
federal probation officers in three districts. 
Future studies will seek to replicate the find- 
ings of previous research (Paparozzi and 
Gendreau, 2005) demonstrating that officers 
with a balanced professional approach real- 
ized lower rates of technical violations and 
rates of arrest than did officers with either law 
enforcement or social worker orientations. 


Previous Research 


Officer Attitudes 


In the past, probation and parole officer 
attitudes have typically been investigated as 
they relate to role orientation (Glaser, 1969; 
Klockars, 1972; Ohlin, Piven, and Papenfort 
1956; O'Leary and Duffee, 1971; and Rowan, 
1956), role conflict (Clear and Latessa, 1993; 
Erwin and Bennett, 1987; Hardyman, 1988; 
and McCleary, 1978), goal orientation (Ross 


and Johnson, 1997), job stress (Whitehead 
and Lindquist, 1989), sources of role orien- 
tation (Clear and Latessa, 1993; Sluder and 
Reddington, 1993), and attitudes towards use 
and carrying of weapons (Sluder, Shearer, 
and Potts, 1991). Officers’ attitudes toward 
their roles and their goal orientation have also 
been linked to officer behaviors to determine 
if an individual officer’s preferences influ- 
ence his or her treatment of clients (Clear 
and Latessa, 1993; Dembo, 1974; Stichman, 
Fulton, Latessa, and Travis, 1997a). There has 
been, however, a dearth of research explor- 
ing officer attitudes and behaviors and their 
effects on probationer/parolee outcomes. As 
Fulton et al. (1997) suggest, if further training 
may modulate an officer’s attitudes towards a 
more balanced approach or towards greater 
orientation towards treatment, the question 
of what effect these attitudes may contribute 
to officer and offender behavior becomes 
increasingly important. 


Officer Attitudes and Performance 


Although there have been numerous studies 
on different types of probation officer atti- 
tudes and behaviors, research on how attitudes 
and role preferences translate into correctional 
practice is limited. Wright (1998) found that 
officers may tend to hold more views sup- 
portive of offender reform than offender 
control. These views may be important to 
probation and parole because they may affect 
how an officer reacts to offender behaviors, 
including violations of supervision. Clear and 
O'Leary (1983) found that attitudes of author- 
ity and assistance were significantly related 
to both the type and the number of supervi- 
sion objectives that officers set for clients. 
Research by Katz (1982) and Duffee (1975) 
shows that attitudes may be related to certain 
behaviors, particularly an officer’s decision 
to recommend revocation. Dembo’s (1972) 
findings supported this link—officers with a 
low reintegrative score, representing a greater 
punishment orientation, were more likely to 
take formal action on violations and to rec- 
ommend return to prison. Dembo concluded 
that officer orientations not only affect job 
behavior, but they may affect case outcomes 
as well. This finding is particularly important 
given the degree of agreement between officer 
recommendations at sentencing and actual 
case disposition (Hagan, 1975; Walsh, 1985). 
In contrast to the notion that individ- 
ual attitudes impact behaviors, research 
conducted by Erwin and Bennett (1987), 
Clear and Latessa (1993), and Stichman et 


al. (1997b) demonstrates that despite clear 
role preferences, officers are able to perform 
tasks of both assistance and control. Clear 
and Latessa (1993) attempted to clarify the 
relationship between role attitude and role 
performance through measuring probation 
officers’ attitudes toward role performances 
and by giving the officers case scenarios for 
which they selected the supervision tasks they 
considered to be important for their clients. 
Stichman et al. (1997b) similarly found that 
although officers in intensive supervision 
programs (ISP) may have different attitudes 
from regular officers, these attitudes were only 
somewhat related to their behaviors. These 
studies suggest that officers, regardless of 
their personal role orientation, could perform 
control or treatment tasks according to the 
policies of the program. 


Officer Attitudes and Outcomes 


Conspicuously missing from the literature 
on officer role orientations in probation and 
parole is a discussion about the influence 
of various orientations on client recidivism. 
Perhaps the most relevant and recent research 
on philosophical groundings and effectiveness 
in probation and parole comes from the inter- 
est in intermediate sanction programs (ISPs). 
Research on ISPs in the past has shown that 
surveillance-oriented approaches are not effec- 
tive in reducing recidivism. Many ISP policies 
either explicitly or implicitly encourage author- 
itative supervision strategies. Although there 
is often an increase in supervision activities 
for offenders on ISP, there is rarely a coincid- 
ing increase in intensity of services. Harris 
(1987:21) notes that this emphasis on control 
has caused field agents to “become avowed 
enemies of their charges, operating...to incar- 
cerate, and as urine takers, money collectors, 
compliance monitors, electronic surveillance 
gadget readers, and law enforcers.” 

Although intervention and services are 
included in their programs, most ISPs empha- 
size surveillance and enforcement. Many 
ISP evaluations demonstrate an increase in 
technical violations for ISP offenders as com- 
pared to offenders placed in other sentencing 
options, but there are no significant differ- 
ences between the groups in new arrest rates 
(Erwin, 1987; Petersilia and Turner, 1993; 
Wagner and Baird, 1993). As stated previ- 
ously, Dembo (1972) found that officers who 
had a greater punishment orientation were 
more likely to take formal action on viola- 
tions and to recommend return to prison. 
As ISP policies encourage more authoritative 
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supervision strategies, it is possible that these 
officers could have more surveillance-oriented 
attitudes, which in turn may lead to higher 
technical violation and revocation rates. 

An evaluation of an ISP operated by the 
New Jersey State Parole Board is particularly 
instructive concerning a link between officer 
attitudes, roles, and ISP outcomes (Paparozzi, 
1994). As further evidence of a relationship 
between treatment and recidivism reduction, 
Paparozzi found that ISP subjects received 
twice as many treatment referrals as a matched 
group of offenders on traditional parole super- 
vision and had a 20 percent lower recidivism 
rate. Paparozzi also examined the relationship 
between officer attitudes, roles, and parole 
outcomes. Based on a survey of officer atti- 
tudes, Paparozzi categorized the ISP officers as 
social work officers, balanced approach offi- 
cers, and law enforcement officers. Recidivism 
data for each category revealed that the social 
work officers had significantly higher rates 
of new arrests and lower rates of technical 
violations, the law enforcement officers had 
significantly higher rates of technical viola- 
tions and lower rates of new arrests, and the 
balanced approach officers had lower rates of 
both technical violations and new arrests. 

Paparozzi concluded that a balanced 
approach to supervision that utilizes the 
full range of potential probation and parole 
activities—intervention, surveillance, and 
enforcement—is essential to meeting both the 
short-term objectives of behavioral change 
and long-term objectives of reduced recidi- 
vism. Additional support favoring the balanced 
approach as the most effective in terms of recid- 
ivism was found by Stichman et al. (1997b). 


Changing Officer Attitudes 


If officer attitude ends up being related to 
offender outcomes and recommendations for 
prison, the policy implications are significant, 
particularly for efforts to change officer atti- 
tudes. However, if officer attitudes are fixed, the 
utility of the findings linking officer attitude to 
offender performance becomes somewhat lim- 
ited. Research by Fulton et al. (1997) indicates 
that officer attitudes can be changed by training 
sessions. In this research, Fulton et al. (1997) 
measured the attitudes of officers who under- 
went comprehensive training and development 
activities and compared them to officers who 
did not participate in such activities. The 
training and development activities focused 
on effective interventions, objectives-based 
case management, and risk/needs assessment. 
The officers who underwent the training and 


TABLE 1. 

Distribution of Respondents Across District 

District N 
Massachusetts! 46 27 
Michigan-Eastern* 65 38 
North Carolina-Middle* 59 35 


' 3 respondents did not complete the OOQ and 1 did not complete the SAS 
* 2 respondents did not complete the OOQ and 1 did not complete the SAS 


* 1 respondent did not complete the SAS 


development were more likely to have attitudi- 
nal scores more supportive of the rehabilitative 
function of probation and were more likely 
to support activities that promote behavioral 
change rather than just surveillance. 

The current study provides the prelimi- 
nary results of a study in the federal system 
that links the responses of officers on two 
surveys to client outcomes. This study covers 
the methodology employed in administering 
the surveys and the results of those sur- 
veys. Investigations regarding the relationship 
between officer responses and caseload out- 
comes will be forthcoming. 


Method 
The researchers asked three districts 
(Massachusetts, the Eastern District of 


Michigan, and the Middle District of North 
Carolina) to participate in the officer sur- 
vey. All three districts have been involved 
in implementing EBP in their districts, have 
shown interest in understanding the impact 
of officer orientation on supervision, and 


. have a history of surveying officers for vari- 


ous purposes. All officers, probation officer 
assistants, supervisory probation officers, 
deputy chiefs, and chiefs were asked to com- 
plete two surveys, the Officer Orientation 
Questionnaire (see Paparozzi & Gendreau, 
2005) and the Staff Attitude Survey (see 
Fulton et al., 1997). 

The researchers used a modified Dillman 
method (Dillman, 2000) to conduct the sur- 
veys. Each chief, through a detailed email, 
advised his or her staff that the district was 
participating in the research. The staff was 
informed of the nature of the research and 
the importance of their participation. A week 
later, the chief sent an email to all officers 
asking them to complete the two surveys 
with survey links. Reminder emails were sent 
after one week, and then again after an addi- 
tional three weeks. Two respondents never 
completed the survey. Six others initiated the 


Officer Orientation Questionnaire but failed 
to completely fill in all items. This led to 152 
usable surveys (out of 160) or a 95 percent 
completion rate for the officer orientation 
questionnaire. Only two respondents failed to 
complete the Staff Attitude Survey; thus, 156 
out of 158 officers completed the question- 
naire, for a 98 percent completion rate. 


Results 


As indicated in Table 1, most respondents were 
from Michigan Eastern (the largest district of 
the three), followed by North Carolina Middle, 
and then Massachusetts (the smallest district). 
Table 2 indicates the percentage and number of 
respondents by position. Clearly, the majority 
of respondents were probation officers (61 per- 
cent), followed by probation officer specialists 
(18 percent). Smaller percentages are noted in 
the probation officer assistant category and the 
administrative groups. 


TABLE 2. 


Distribution of Respondents Across 
Position 


Position N % 
Probation Officer Assistant 6 4 
Probation Officer? 97 61 
Probation Officer Specialist® 28 18 
SUSPO 20 13 
Deputy Chief’ 5 3 

Chief® 2 1 


* Two respondents did not complete the OOQ 
and one did not complete the SAS. One officer, 
not represented in these numbers, did not initiate 
either survey. 

° Three respondents did not complete the OOQ. 


” One respondent did not complete the OOQ and 
one respondent did not complete the SAS. 


* One chief, not represented in these numbers, did 
not initiate either survey. 
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The Officer Orientation Questionnaire 


The officer orientation questionnaire (OOQ) 
was developed by Dembo (1972). The sur- 
vey comprises 24 items rated on a scale of 1 
to 7, indicating the officer's agreement with 
statements that represent two extremes of a 
continuum. For example, one item has the 
extremes of “the causes of crime are located in 
factors internal to the offender” and “the causes 
of crime are to be found in factors external to 
the offender.” Officers must rate where their 
belief falls on the continuum. The possible 
range of scores is 24 to 168. The OOQ was used 
by Paparozzi & Gendreau (2005) and they pro- 
vided the following scoring cutoffs: 24-71 law 
enforcement oriented officers, 72-120 balanced 
officers, and 121-168 social work oriented offi- 
cers. The average score on the OOQ for those 
that completed the survey was 112.75, with a 
standard deviation of 13.55. The range was 61 
to 142. There was very little variation in the 
average scores by district. An F-test confirmed 
that the differences from one district to another 
were not statistically significant. 

Table 4 contains the distribution of officer 
orientation categories based on their overall 
scores and cutoffs used by Paparozzi and 
Gendreau (2005). As can be seen from Table 
4, one respondent was categorized as law 
enforcement oriented, 115 were identified as 
balanced and 48 were identified as social work 
oriented (1 percent, 70 percent, and 29 percent 
respectively). Federal officers in this sample 
are apparently balanced; almost 70 percent 


to measure officer attitudes. The items are 
designed to measure attitudes about the goals 
of supervision, officer roles, and supervision 
strategies. For example, one item states “your 
primary concern as a probation officer is to 
monitor offender compliance” (a score of 1) 
versus “rehabilitate the offender” (a score of 7). 
Respondents are instructed to circle the num- 
ber on the scale that best describes them. The 
scale was modified from a range of 1 to 6, to a 
range of 1 to 7, to allow respondents the option 
to choose a neutral score within the middle of 
the range. The SAS includes two scales. The 
first, the Subjective Role Scale, has items that 
focus on what officers do. The second scale, the 
Strategy Scale, evaluates how officers perform 
their functions (Fulton et al., 1997). 


TABLE 3. 


Table 6 provides the descriptive statistics 
for the scale’ scores and the average response 
across the 33 items. All scales were coded so 
that higher numbers indicate a social work 
or intervention perspective rather than law 
enforcement. The scale scores were slightly 
higher than those reported by Fulton et al.; 
however, that could be because we lengthened 
the possible responses on our survey from 1 
through 6 to 1 through 7. The overall aver- 
age for the 33 items is 4.57, which is close to 
the middle of the scale (4.6 out of a possible 
7). Again, as with the Officer Orientation 
Questionnaire, the SAS data indicate that 
these officers are balanced. The averages for 
all respondents are presented, as there were no 
differences by district. 


Descriptive Statistics for the OOQ for All Respondents and by District 


District N Mean Standard Deviation Minimum Maximum 
All 152 113.00 12397 61 142 
MA 43 114.19 11.56 85 140 
MI-E 63 112.68 15.20 61 142 
NC-M 46 142,33 13D 83 141 


F(149,2) = 0.236; p = 0.790 


TABLE 4. 


Distribution of Officer Orientation Categories for All Respondents and by District 


al's data were .88 and .78 respectively. 


are scored as balanced; only one percent score _| District Law Enforcement Balanced Social Work 
as law enforcement. In contrast, in a study by N o, %, N %, 
Paparozzi and Gendreau (2005), officers were 
equally distributed across three groups. di . = wi = = 
Additional analysis revealed that scores | MA 0 0 31 72 12 28 
differ significantly by position (see Table 5). | MIE 1 2 41 65 21 33 
Post hoc analysis indicated that probation NC-M 0 0 33 72 3 28 
officers differed significantly from chiefs. It 
is unclear why there is a difference between (4) = 2-602; p = 0.627 
the upper management and line staff. One 
hypothesis may be that all three districts are apie 5. 
heavily involved in EBP efforts, which tend to Descriptive Statistics for the OOQ by Position 
fall in the social work category, and this may 
be the emphasis for the leadership at this time. | District N Mean | Standard Deviation | Minimum | Maximum 
Staff Attitude Survey Probation Officer 6 | 116.83 16.388 95 140 
Assistant 
95 | 110.15 13.299 61 138 
a8 33 semantic differentials that are intended 
oe Probation Officer 25 | 116.32 9.503 93 131 
a * Asa note, the alpha reliability coefficients gener- _| Specialist 
- ated with these data were somewhat smaller than 
those generated by Fulton et al. (1997). The alpha 
reliability scores for these data were .83 and .69 for Deputy Chief 4 125.00 12:723 110 137 
the subjective role and strategies scales respectively, . 


F(150,5) = 3.240; p = 0.008 
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There are variations on the SAS by posi- 
tion (see Table 7). Specialists and supervisors 
have a slightly greater social work orientation 
than probation officer assistants or probation 
officers. Additionally, deputy chiefs and chiefs 
score a bit higher on the survey than do the 
other respondents, indicating more of a social 
work orientation. 


Discussion 


The results from this preliminary analysis 
are encouraging. These data indicate that the 
federal officers who completed these surveys 
are balanced in their approach to their work. 
Such a balanced approach has been shown 
to yield lower rates of technical violations 
and lower rates of rearrests in prior studies 
(Paparozzi, 1994), and bodes well for the 
federal probation system as these are two pri- 
mary outcome measures that are considered 
critical. This finding is also consistent with 
recent data that have shown that the rearrest 
rates of federal offenders are comparatively 
low, ranging from 12 percent at year one, 
18 percent at year two, and 24 percent after 
year three (Baber, 2010), outcomes that 
are generally lower than those seen in state 
systems. Finally, these results are consistent 


TABLE 6. 


with recent survey data from offenders (see 
Offender Survey article, this issue of Federal 
Probation), in which offenders had very 
favorable impressions of their officers, with 
94 percent describing their officers as “firm 
but fair” Taken together, they present a posi- 
tive impression of federal supervision. 


Limitations 


A significant limitation to this study is that it 
only included three of the 94 districts in the 
system. The small number of districts cer- 
tainly precludes generalizations beyond the 
three districts. In addition, these districts have 
been involved in implementing evidence- 
based practices and decision making for many 
years, which calls into question the degree to 
which they represent the typical district in the 
federal system. The second major limitation 
is the fact that we have yet to link the officer 
responses to caseload outcomes. As it now 
stands, this study is merely one of many that 
seek to understand the orientation of proba- 
tion officers, albeit with a unique sample of 
federal officers. However, the linking of officer 
responses on these surveys to officer caseload 
outcomes will be completed within the next 
six months. 


Descriptive Statistics on Subjective Role Scale, the Strategies Scale, and the Average 


Item Response for All Respondents 


Scale N Mean Standard Deviation Minimum Maximum 
Subjective 168 29.64 4.84 15 42 
168 4.56 0.58 3.36 6.33 
Rating 

TABLE 7. 


Descriptive Statistics for the Average Item Response on the SAS by Position 


District N Mean Standard Deviation Minimum Maximum 
Probation 

Officer 6 4.62 0.46 4.03 5S 
Assistant 

Probation 96 4.41 0.52 3.36 6.00 
Officer 

Probation 

Officer 28 4.87 0.64 4.00 6.33 
Specialist 

SUSPO 20 4.76 0:57 343 5.76 
Deputy Chief 4 4.97 0.50 4.55 5.67 
Chief 2 0.11 5.30 


F(150,5) = 4.738; p = 0.000 


Conclusion 


As discussed earlier, even the best evidence- 
based principles must be “washed through” 
the filters of staff values and professional 
orientation. Arguably, balanced officers create 
relationships with offenders that are neither 
indulgent of anti-social attitudes and noncom- 
pliance nor authoritative and heavy-handed. 
Balanced officers utilize both ends of the 
extreme, likely responding most appropri- 
ately to offender behavior and providing 
a pro-social role model for offenders. As 
offenders generally have more contact with 
officers than with any referral agent over 
the course of three to five yéars of supervi- 
sion, it is critical that agencies tap into the 
potential effectiveness of these interactions. 
Combining the balanced approach with the 
evidence-based programs and principles that 
continue to emerge (i.e., Bonta et al., 2008) 
will likely result in significant increases in 
positive outcomes. As community correc- 
tions increasingly recognizes the critical role 
of officers as interventionists, rather than 
simply brokers of services, it is important that 
we fully understand the impact that officer 
attitudes and orientation may have on the 
supervision process. 
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AS COMMUNITY CORRECTIONS pro- 
fessionals, should we care about how offenders 
perceive us? The research demonstrates that 
focusing simply on the officer/offender rela- 
tionship generally does not reduce offending 
(Andrews & Bonta, 2003; Trotter, 1996). 
However, in one critical study, officers who 
made use of open and warm communication 
skills and demonstrated mutual respect with 
offenders had greatly improved recidivism 
outcomes—the offenders they supervised 
were half as likely to return to prison as those 
offenders who were supervised by officers 
who did not use those skills (Trotter, 1996). 
The importance of the officer/offender rela- 
tionship was first identified as part of a larger 
set of skills called core correctional practices 
(Andrews and Kiessling, 1980; Andrews and 
Carvell, 1998). Core correctional practices 
include: 1) the effective use of authority, 2) 
anti-criminal modeling and reinforcement, 
3) problem solving, 4) use of community 
resources, and 5) the quality of interpersonal 
relationship between staff and client. Andrews 
and Kiessling argued that a positive rela- 
tionship between the officer and offender is 
necessary for other core correctional practices 
to be effective; however, a relationship alone 
does not reduce recidivism. According to 
Dr. Faye Taxman, “the glue of this [change] 
process is deportment or the manner of being 
between the offender and officer” (Taxman, 
2002). This positive relationship is ‘necessary 
but not sufficient to bring about behavioral 
change” (Spiegler and Guevremont, 1998). 


How are we to gauge our relationships with 
offenders and determine whether the relation- 
ships support the behavioral change process? 
One easy approach is through offender sur- 
veys that directly inquire about the offenders’ 
experiences. Client “satisfaction surveys” have 
long been associated with effective correc- 
tional programs (Gendreau and Andrews, 
1994 & 2002; Lowenkamp, 2004; Lowenkamp, 
Latessa, & Smith, 2005). Given the impor- 
tance of administering client satisfaction 
surveys, the Office of Probation and Pretrial 
Services (OPPS), Administrative Office of the 
US. Courts developed a short survey to be 
completed by offenders on post-conviction 


supervision (Whetzel, 2010). 


Survey Administration 


In 2009 two districts, New York Western 
(NYW) and Kentucky Western (KYW), 
piloted an offender survey developed by 
OPPS. The OPPS survey was based in part 
on input from various districts that had previ- 
ously surveyed their offenders. In the OPPS 
survey, offenders were asked to respond to 
each of the following statements: 

1) My probation officer is firm but fair 

2) My probation officer is a positive role 

model 


3) My probation officer helps me to learn 
how to solve my problems 

4) My probation officer helps to arrange 
various services for me 

5) My probation officer assists me in 
securing employment 


Jay Whetzel 

Christopher T. Lowenkamp 
Probation Administrators 
Office of Probation and Pretrial Services 
Administrative Office of the U.S. Courts 


6) My probation officer communicates 
with me openly and respectfully 
7) My probation officer helps to motivate 
me 
8) My probation officer includes my fam- 
ily in my supervision 
9) Being on supervision has had a posi- 
tive effect on my life 
10) My probation officer frequently meets 
me in my community 
11) My probation officer is involved in 
my substance abuse or mental health 
treatment 
12) When I do the right thing, my proba- 
tion officer acknowledges it 
Offenders had to answer using a five- 
point Likert scale with the following choices: 
Strongly Agree, Agree, Undecided, Disagree, 
and Strongly Disagree. Offenders could also 
provide commentary. To conduct the survey, 
Kentucky Western mailed the questions to 
300 randomly selected offenders; the response 
rate was 30 percent. In New York Western, 
offenders were asked to complete the surveys 
while in the waiting room. Officers were not 
involved in the completion of the surveys. 
In January 2011 the Northern District of 
Texas also conducted the survey using Survey 
Monkey, sending the survey link to 445 
offenders who submit their monthly reports 
electronically. In all, 372 offenders completed 
the survey: 97 from KYW, 146 from NYW, 
and 129 from TXN. 
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Results 


All three districts were pleasantly surprised 
by their results. Taken together, the survey 
data showed that 96 percent of the offenders 
strongly agreed or agreed that their officers 
were “firm but fair,’ a term coined by Andrews 
and Kiessling that summarized the effective 
use of authority. The qualitative data, ice., 
offenders’ commentary, was also overwhelm- 
ingly positive. While some offenders used the 
survey as an opportunity to criticize their offi- 
cers, the vast majority expressed appreciation 
for the support their officers were providing. 

TX-N has made use of the information 
in several different ways, including sharing 
it with judges and preparing PowerPoint 
slideshows of the outcomes to show on tele- 
vision screens in the offender waiting area. 
Where an offender provided specific positive 
comments and identified the officer, their 
comments were sent to the officer and the 
officer's supervisors. According to Northern 
District of Texas Deputy Chief Jerry Ritchie, 
that was “quite a positive shot in the arm for 
many officers.” The survey data will also be 
used in the district's external website and in 
the district annual report. 


The aggregate data from all three districts 
(KYW, NYW, TXN) is presented below. 

An impressive 96 percent of offenders 
indicated that their officer was firm but fair. 
Consider the following offender comments: 
“my PO is very good at what he does and has 
seen me go through a lot during this time. He 
has been proud of me when I’ve done well and, 
when I’ve messed up, he was stern but didn't 
treat me poorly or make me feel bad about 
myself”; and “I just want to say that my PO is 
by the book, takes no crap, but he truly cares.” 
In the research literature, offenders on super- 
vision are considered a type of "involuntary 
client,” that is, they are receiving assistance 
and direction from the probation officer, but 
not of their own volition (Trotter, 1996). In 
order to establish a productive “therapeutic 
alliance” between the offender and the proba- 
tion officer, officers must be authoritative (but 
not authoritarian) and clearly define their own 
and the offenders’ roles (Skeem et al., 2007). 
The effective use of authority requires offi- 
cers to “explicate the formal rules associated 
within the correctional setting such that they 
are made more visible, understandable, and 
unambiguous” (Dowden and Andrews 2004). 


TABLE 1. 
Responses to Survey Questions 
Strongly Strongly 
Agree | Agree | Undecided| Disagree | Disagree| Median 
My PO is firm but fair 61% 35% 3% 1% 1% 1 
sted bey is a positive role 46% | 43% 7% 3% 1% 2 
ea a how | 33% | 48% | 13% 5% 1% 2 
My PO helps to arange 31% | 49% | 15% 3% 2% 2 
various services for me 
22% | 47% | 22% 7% 2% 2 
My PO communicated with 59% 36% 2% 1% 1% 1 
me openly and respectfully 
35% | 48% | 13% 3% 2% 2 
My my | | | 15% 8% 2% 2 
in my supervision 
Being on supervision has 
had a positive effect on 35% 39% 15% 6% 5% z 
my life 
When | do the right thing 
my PO acknowledges it 48% 37% 10% 4% 1% 2 


Eighty-nine percent of offenders “strongly 
agreed” or “agreed” that their probation offi- 
cers were positive role models, i.e., offering 
what Andrews and Kiessling would call “anti- 
criminal modeling.” The concept of “PO as 
offender role model” has not been tradition- 
ally discussed or integrated into new officer 
training, where officer behavior is typically 
discussed in relation to professional conduct 
as officers of the court, not as models for 
anti-criminal behavior. Consider these com- 
ments: “My experience with the probation 
officers has been very positive. Good people 
with a level head and common sense. The 
interaction that I have experienced has been 
a very positive one. They have been courte- 
ous, friendly and responsive”; and “My PO 
sets an example that all probation officers 
should follow: returns phone calls promptly, 
shows concern, pleasant voice. She is a good 
example....” This is critical, as Trotter (1996) 
notes the impact that prosocial modeling has 
on involuntary clients. 

Offenders are indeed involuntary clients 
and are frequently advised that failure to 
participate in the supervision process will 
lead to negative consequences. In such a 
relationship, it is all too easy for officers to 
assert their authority excessively and to fail 
to treat offenders with respect. Consistent 
with core correctional practices, officers must 
“avoid interpersonal domination or abuse” 
(Dowden and Andrews, 2004). Fortunately, 
over 95 percent of offenders in these surveys 
reported that their officers communicate with 
them openly and respectfully. What we may 
fail to realize, however, is that such a rapport 
with an offender is a necessary precondition 
for enhancing an offender's intrinsic motiva- 
tion to change. The following offender quote 
reflects this effectively. 

I have had several officers through- 
out the years and I must say that 
personally, it makes a huge differ- 
ence in my attitude and behavior, 
when I am treated like an equal, or 
at least not talked down to... When 
I am treated like a productive and 
equal member of society, it makes 
me want to act like one, and make 
my PO proud. 

The data show that offenders strongly 
agree or agree that officers assist with prob- 
lem solving (81 percent), motivate them (83 
percent), and acknowledge their successes (85 
percent). “My PO has helped me learn how 
to solve problems, motivates me and always 
acknowledges when I do the right thing.” All 
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three of these behaviors reflect the use of core 
correctional practices. 

The use of offender surveys marks a 
new level of engagement in the supervision 
process. Understanding that offenders’ prob- 
ability of success is enhanced through the use 
of evidence-based core correctional practices 
compels us to integrate them into our every- 
day involvement. The new STARR program 
(Supervision Techniques Aimed at Reducing 
Re-Arrest) is built on the same core correc- 
tional practices highlighted in the offender 
survey. To become an evidence-based system, 
perhaps the first lesson we must learn is that 
our effectiveness as change agents depends, 
to some degree, upon us, on who we are and 
how we treat offenders. The offender survey is 
a good place to start. 
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[Editors Note:The following article was accepted 
after under going a double-blind peer review 
process, in recognition of the critical importance 
of the Post Conviction Risk Assessment tool 
to the federal probation systems adoption of 
evidence-based practices. ] 


THE UNITED STATES PROBATION system 
was created in 1925 by the Federal Probation 
Act. This Act gave the U.S. Courts the power 
to appoint federal probation officers and the 
authority to sentence defendants to probation 
instead of a prison term. One of the primary 
functions of federal probation is to supervise 
convicted offenders who are sentenced to a 
term of probation or a term of supervised 
release following a period of imprisonment, 
and offenders released early from prison on 
parole or mandatory release by the U.S. Parole 
Commission or military authorities. 

The federal probation and pretrial services 
system is organized into 94 districts within 11 
regional circuits and operates under a decen- 
tralized management structure. As a result 
of being decentralized, each district oper- 
ates with a great deal of autonomy; however, 
despite this autonomy, the system maintains 
cohesion through the Administrative Office 
of the U.S. Courts (AO). The AO serves as the 
administrative headquarters for this decen- 
tralized system and develops national policies 
that help districts in their efforts to protect the 
community and reduce recidivism. 

During the past two decades, advance- 
ments in social science research, the need to 


use resources more efficiently and effectively, 
and increased expectations to reduce recidi- 
vism have sparked a major philosophical shift 
in the field of probation. Although probation 
officers are still required to monitor offender 
behavior and report noncompliance to the 
court, the general focus has shifted to reduc- 
ing future criminal behavior (Alexander & 
VanBenschoten, 2008). Arguably, the best 
chance for reducing recidivism occurs when 
officers not only have a reliable way of dis- 
tinguishing high-risk offenders from low-risk 
offenders but also can intervene in the crimi- 
nogenic (crime supporting) needs of high-risk 
offenders (Andrews et al., 1990; Lowenkamp 
& Latessa, 2004; Bonta & Andrews, 2007; 
Campbell, French & Gendreau, 2007). For 
federal probation, this has meant looking for 
more effective ways to manage offenders by 
predicting their potential to reoffend and/or 
their potential dangerousness to the commu- 
nity (Walklate, 1999). 

This article explains the process the AO 
used to develop a risk assessment instrument 
for use with its post-conviction supervision 
population. We provide a brief overview of 
the principles of effective classification and a 
summary of the evolution of risk assessments 
and then explain why the AO chose to create 
its own risk assessment instrument rather 
than use an existing instrument. However, 
the primary purpose of the article is twofold: 
(1) To present the methodology and results 
produced in the development of the Post 
Conviction Risk Assessment (PCRA) tool, and 
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(2) to discuss limitations of the PCRA as well 
as future developments. 


Principles of Effective 
Risk Classification 


In general terms, the principles of effec- 
tive risk classification refer to the prediction 
or identification of offenders most likely to 
violate the law or conditions of supervision 
during a period of criminal justice supervi- 
sion, the identification of factors that can be 
influenced to change the likelihood of recidi- 
vism, and the acknowledgement of factors that 
might influence the benefits of a particular 
service (Van Voorhis & Brown, 1996). Risk of 
recidivism, criminogenic need, and general 
responsivity are three of the primary prin- 
ciples of effective classification (Andrews et 
al. 1990). The fourth principle, professional 
discretion, targets the professional's ability to 
look beyond the application of the first three 
principles when circumstances indicate a need 
to do so (Gottfredson, 1987). 

The principles of effective risk classifica- 
tion suggest that agencies should use actuarial 
assessment tools to identify dynamic risk fac- 
tors, especially in high-risk offenders, while 
also identifying potential barriers to treat- 
ment (Bonta & Andrews, 2007; Latessa et 
al., 2010). Actuarial risk assessments rest on 


* 


Corresponding author. 

** For questions related to analysis. 

+ For questions related to automation, 
certification, training and implementation. 


te inf lg 


September 2011 


CONSTRUCTION AND VALIDATION OF PCRA 17 


three factors: (1) certain individual character- 
istics and behaviors are statistically predictive 
of future involvement in criminal behavior; 
(2) the more risk factors an offender has, 
the greater the likelihood of future criminal 
behavior; and (3) when properly validated 
and administered, actuarial risk predictions 
are more accurate than clinical predictions 
(Meehl, 1954; Sawyer, 1966; Gottfredson, 
1987; Andrews and Bonta, 1994). Andrews 
and Bonta (1998) argue that it is the combined 
assessment of risk and need that improves the 
ability to predict who is likely to offend and 
outlines what interventions should take place 
to reduce risk and subsequently recidivism. 


Brief History of Risk 
Assessment Tools 


Purpose of a Risk Assessment Tool 


The assessment of offenders has long been 
acknowledged as a necessary component for 
criminal justice practitioners who are respon- 
sible for assessing and managing offenders. In 
the field of probation, the primary purpose 
for using a risk assessment tool is to help keep 
communities safe from offenders who are most 
likely to reoffend. Although security was the 
primary reason for the development of risk 
assessment instruments, the ability to classify 
offenders at the appropriate risk level is also 
beneficial. Consequently, risk assessment tools 
help probation officers identify which offenders 
need intensive interventions and what needs 
should be targeted by the interventions. 


Evolution of Risk 
Assessment Instruments 


The evolution of risk assessment is described 
as following a generational path that started 
with the most basic form of assessment and 
has progressed to a more complex form of risk 
assessment (Bonta & Wormith, 2007). Each 
generation utilized the best available methods 
to predict the risk of recidivism and then 
applied the results of the assessment to super- 
vision strategies. This tradition continues 
today, with researchers continually refining 
their understanding of criminal behavior and 
the associated enhancements to risk/needs 
prediction tools (VanBenschoten, 2008). 


First generation 


For most of the 20th century, professional 
judgment or intuition was the most common 
method used to predict criminal behavior. 
This form of assessment involved an unstruc- 
tured interview with the offender and a review 


of official documentation (Bonta, 1996; Van 
Voorhis & Brown, 1996; Andrews & Bonta, 
2006; Connolly, 2003). Guided by their own 
professional training and experience, proba- 
tion officers and clinical professionals made 
judgments about who required enhanced 
supervision or correctional programming 
(Bonta & Andrews, 2007). One of the inherent 
weaknesses of such an unstructured process 
is the lack of a quantitative way to determine 
how decisions are reached, which leads to a 
lack of consistency and agreement resulting in 
low inter-rater reliability (O'Rourke, 2008). In 
other words, the same interview conducted by 
different interviewers could net dramatically 
different results; therefore, the conclusions 
and recommendations regarding the offender 
could vary depending on the interviewer 
(Wardlaw & Millier, 1978; Monahan, 1981; 
Van Voorhis & Brown, 1996). 


Second generation 


Although second-generation risk tools have 
been available since the late 1920s, it was not 
until the 1970s that the assessment of risk 
began to depend more upon actuarial, evi- 
dence-based science and less on professional 
judgment and intuition. Second generation 
risk assessments are often referred to as actu- 
arial methods (O’Rourke, 2008). Actuarial risk 
assessments consider individual items (e.g., 
history of substance abuse) that have been 
demonstrated to increase the risk of reoffend- 
ing and assign these items quantitative scores 
(Bonta & Andrews, 2007). Burgess (1928) 
established the first of these models. In the 
Burgess method, each variable in the model 
can be scored as a “point,” and the prediction 
is based on the aggregate number of points 
assigned to an offender (Connolly, 2003). For 
example, the presence of a risk factor may 
receive a score of one and its absence a score 
of zero. The scores on the items can then be 
summed—the higher the score, the higher the 
risk that the offender will reoffend (Bonta & 
Andrews, 2007). This technique gives equal 
weight to all predictors, even though there 
may be unequal effects. There is little research, 
if any, indicating that more complex (i.e., 
weighted) scoring methods produce better 
prediction than simple (i.e., unweighted) 
methods (Gottfredson 1987). 


Third generation 


Recognizing the limitations of second-genera- 
tion risk assessment, research began to develop 
in the late 1970s and early 1980s on assess- 
ment instruments that included dynamic risk 


factors (Bonta & Wormith, 2007). The third 
generation of assessment is commonly referred 
to as risk-need assessments (Andrews & Bonta, 
1995; Bonta & Andrews, 2007). These instru- 
ments combined the static predictor variables 
of the second-generation instruments with 
dynamic criminogenic need items (e.g., pres- 
ent employment, criminal friends, and family 
relationships) that were sensitive to changes in 
an offender's circumstances (Connolly, 2003; 
Bonta & Andrews, 2007). Third-generation 
risk assessment tools exceed statistical risk 
prediction by adding the element of need 
identification. As previous instruments assisted 
in decision-making regarding supervision 
conditions, third-generation assessments help 
identify areas that require intervention to miti- 
gate recidivism risk while under supervision 
(Van Voorhis & Brown, 1996). 


Fourth generation 


The last few years has seen the introduction 
of fourth-generation risk assessment instru- 
ments. These new risk assessment instruments 
go beyond the third-generation risk-need 
assessments. Not only do fourth-generation 
instruments include risk-need assessments, 
they also assess a broader range of risk factors 
along with responsivity factors important to 
treatment for integration into the case man- 
agement plan (Bonta & Andrews, 2007; Bonta 
& Wormith, 2007). Some examples of respon- 
sivity factors include reading and cognitive 
abilities, race, gender, motivation to change, as 
well as external factors such as treatment set- 
ting and counselor characteristics (Andrews et 
al., 1990; Bonta & Wormith, 2007). One other 
aspect of fourth-generation risk assessments 
is the attempt to explicitly link identified 
needs with supervision and treatment services 
(Bonta & Wormith, 2007). 


Post Conviction Risk 
Assessment Tool 


Actuarial risk assessments are not new to the 
federal probation system; in fact, they have 
been part of the supervision process since 
the early 1980s. To better assist probation 
officers in identifying high-risk offenders 
and intervening in their criminogenic needs, 
the AO chose to develop a risk assessment 
instrument tailored specifically to its popula- 
tion of offenders. The Post Conviction Risk 


Assessment (PCRA) is an actuarial risk and 
needs assessment tool developed from data 
collected on federal offenders who started a 
term of supervision between October 1, 2005 
and August 13, 2009. This tool is designed to 
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target treatment interventions prioritized by 
risk, need, and responsivity. 


How the PCRA Came into Existence 

In the Strategic Assessment of the Federal 
Probation and Pretrial Services System (hereaf- 
ter cited as IBM, 2004), the authors identified 
shortcomings with the AO’s use of the Risk 
Prediction Index (RPI).' One of the concerns 
expressed by the authors was the RPI’s static 
nature, which causes a disconnection between 
the risk score and case management (IBM, 
2004). Put another way, if an offender's risk to 
recidivate changes during the course of super- 
vision, the RPI does not reflect this change; 
therefore, officers are not able to consistently 
and effectively interpret those changes and 
provide the proper supervision response. 

To address the RPI’s shortcomings, the 
Strategic Assessment recommended that 
the AO research other data-driven supervi- 
sion tools (IBM, 2004). The desire to meet 
this recommendation, coupled with emerg- 
ing criminal justice literature about more 
advanced risk assessment tools, influenced 
the AO to develop its own Research to 
Results (R2R) effort. During the R2R effort, 
16 of the 94 federal probation districts were 
awarded funding to implement evidence- 
based practices* into their districts. Of those 
16 districts, five districts chose to use a com- 
mercially available risk and needs tool to 
conduct risk assessments. In addition, AO 
staff members met with developers of three 
commonly used off-the-shelf risk/needs tools 
(LS/CMI, COMPAS, RMS)’ to better under- 
stand the advantages and disadvantages of 
each tool. 

Since the federal criminal justice system 
represents a distinctive population and since 
specific trailer assessments for special needs 
populations (such as sex offenders) are also 


The RPI uses 8 largely static questions to 
determine the risk that an offender will recidivate 
during his or her term of supervision; the results are 
intended to assist officers in creating the offender's 
initial supervision case plan. 


Districts were required to submit a proposal, 


which included a budget, outlining an area of 


evidence-based practices (EBP) they wanted to 
implement. The areas of EBP available were risk 
assessment, cognitive behavioral interventions, 
motivational interviewing, and other. The “other” 
category was open and districts that chose this 
option tended to use it for drug courts and work- 
force development. 

LSI (Level of Service Inventory), COMPAS 
(Correctional Offender Management Profiling for 
Alternative Sanctions), RMS (Risk Management 
Services). 


required, it became obvious that more flex- 


ibility would be needed. At the conclusion of 


the experimentation and information gather- 
ing stage, the AO assembled a panel of experts 
to examine the options of purchasing a com- 
mercially available tool or building a new tool. 
After much discussion, the consensus of the 
group was to build a new tool with data spe- 
cific to federal probation. 


Construction and Validation of 
the PCRA 


Methods 


Data used to construct and validate the PCRA 
came from federal presentence reports (PSR), 
existing risk assessments, criminal history 
record checks, and PACTS.’ Criminal history 
records or rap sheets were used to identify 
any new arrest after the start of supervision. 
The five R2R districts that were using a com- 
mercially available risk assessment tool were 
asked by the AO to provide data to assist in the 
development of the PCRA. Each district pro- 
vided a list of offenders who had received an 
assessment using an off-the-shelf risk predic- 
tion instrument and who also had a completed 
PSR. In total, the five districts submitted a list 
of 4,746 offenders, from which 479 cases were 
randomly selected.° Districts were then asked 
to provide rap sheets on the randomly selected 
cases. PACTS was the main source of data for 
scored elements on the PCRA; it included data 
on roughly 100,000 offenders. 


Data Elements 


There are two sets of items included on 
the PCRA: scored and not scored. The first 
set of items are rated and scored and thus 
contribute to an offender's risk score. Rated 
and scored items used to develop the PCRA 


were based on prior research in the area of 


predicting criminal behavior (for example 


* PACTS (Probation/Pretrial Services Automated 


Case Tracking System) is an electronic case man- 
agement tool used by probation and pretrial 
services officers in all 94 federal districts to track 
federal defendants and offenders. At the end of each 
month, districts submit case data into a national 
repository that is accessible to the Administrative 
Office of the U.S. Courts (AO), Office of Probation 
and Pretrial Services. 

One district was not an R2R district but had been 
using a commercially available risk assessment tool 
(RMS) longer than the other four R2R districts. 


Districts were initially informed that 100 cases 
from each district would be randomly selected, 
but one district only permitted 10 percent of their 
cases to be selected, which limited their sample to 
64 cases. 


Gendreau, Little, & Goggin, 1996; Simourd 
& Andrews, 1994; Hubbard & Pratt, 2002; 
Andrews & Bonta, 2006) that were also avail- 
able in PACTS. Based on a review of extant 
research, data elements related to criminal his- 
tory, peer associations, family, employment, 
substance abuse, and attitudes were selected 
from PACTS. As a result of bivariate analyses, 
some interval and ratio variables (e.g., age, 
prior arrests, education, and drug and alcohol 
problems) were collapsed into ordinal mea- 
sures. Multivariate models and completeness 
of data were used to identify the most predic- 
tive and practical data elements to be included 
on the instrument. Variables included on the 
PCRA had a significance level of .10 or below 
(see Table 1). 

The second set of data elements are rated 
but not scored and do not contribute to an 
offender's risk score. These items were iden- 
tified as potentially predictive in a smaller 
sample of offenders from five of the R2R 
districts. With the exception of peer relation- 
ships, which came from the COMPAS and 
RMS, data elements came from the PSR. A 
total of 104 elements were collected from 
the PSR; however, four of those elements 
were personal identifiers (i.e., first name, last 
name, middle initial, and PACTS number). 
Additional rated but not scored items were 
added based on probation officers’ input on 
what data they need to supervise a case (see 
Appendix 1). A total of 29 factors were identi- 
fied as potential predictors and included on 
the assessment. These potential predictors 
were included as “test items” and future analy- 
sis will determine whether these items will 
become rated and scored PCRA items.’ 


Sample 


In order to construct and validate the PCRA, 
the researchers devised three sample groups. 
A construction group was created for the con- 
struction of the instrument, and two validation 
groups’ were created for the validation of the 
instrument. These groups were created using 
an existing analysis file from PACTS data that 
contained 185,297 offenders on probation or 


Due to ongoing data collection, the test items 
have yet to be analyzed. Decisions to include or 
omit test items will be determined by statistical 
significance and by how a test item impacts the 
predictive accuracy of the PCRA. 


8 


Two validation samples were developed in order 
to test the robustness of the instrument. 
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TABLE 1. 
Multivariate Model Predicting Arrest During Initial Case Plan Period (Split Sample Construction Only) 
Variable B SE Wald df Sig Exp(B) 
Community Supervision Violation 343 .052 43.551 1 .000 1.410 
Varied Offending Pattern .226 049 21.416 1 .000 £250 
Institutional Adjustment .227 103 4.848 1 .028 1,255 
Violent Offending 320 .079 16.312 1 .000 1.378 
Unemployed 368 045 66.248 1 .000 1.445 
Poor Work Outlook 322 .061 27.495 1 .000 1.380 
Alcohol Problems 479 .102 22.079 1 .000 1.615 
Lacks Social Support .267 .048 30.673 1 .000 1.306 
Family Problems 191 051 14.278 1 .000 1.210 
Single .097 1 075 1.102 
Not Motivated to Change 383 .050 59.803 1 .000 1.467 
Drug Problems 710 .062 132.195 1 .000 2.033 
Arrest History .149 021 50.543 1 .000 1.160 
Age 383 .033 136.614 1 .0O0O0 1.467 
Educational Attainment 234 045 27.195 1 .000 1.264 
Mental Health Problems .068 . 049 1.920 1 .166 1.070 
Gambling Addiction -.395 .283 1.945 1 .163 .674 
Criminal Associates -.080 .050 2.529 1 BLN? 923 
Weapon Concerns -.086 .064 1.789 1 181 917 
Financial Problems -.070 .078 .806 1 369 .932 
Life Skills Deficiencies -.019 .060 .103 1 .981 
Female -215 .058 13.586 1 .00O0 .807 
Race 3.106 4 540 
Asian .613 1.568 1 1.846 
Black .638 467 1.866 1 172 1.892 
Native American/Eskimo .668 475 V.9F7 1 .160 1.951 
White .683 466 2.145 1 .143 1.980 
Constant -4.540 472 92.691 1 .000 O11 


Model IT*(26) = 1503.78, p < .000; -2LL = 15868.80; Nagelkerke R* = .119 


supervised release.’ The construction group 
was created from data obtained from the initial 


* Data from the analysis file was assembled from 
PACTS and matched with data from the Federal 
Bureau of Prisons (BOP), the U.S. Sentencing 
Commission (USSC), and the Census Bureau. 
Arrest data came from ATLAS (Access to 
Law Enforcement System) and from the FBI's 
Computerized Criminal History (CCH) database. 
Arrest data are current through August 13, 2009. 
Offenders in the analysis file began active post- 
conviction supervision between October 1, 2004 
and August 13, 2009 (see Baber, 2010). Of the 
185,297 offenders in the analysis file, only 103,071 
had criminal histories and other relevant items used 
to construct the PCRA. 


case plan.'° Using a near 50/50 split, data from 
the first case plan was divided into two sample 
groups; one became the construction sample 
and the other became the first validation 
group. One validation group (Validation) was 
taken from the initial case plan the offender 
receives during his or her term of supervision 
and the second validation group was taken 
from subsequent case plans (hence the name 


' As outlined in the Guide to Judiciary Policy, 
Volume 8, Part E, Supervision of Federal Offenders, 
case plans are to be submitted within 30-60 days 
of the start of the offender's supervision term. This 
plan is formally evaluated and modified during the 
sixth month of supervision and updated annually 
for the duration of the supervision term. 


Subsequent Case Plan). Both the construction 
(N=51,428) and validation (N=51,643) groups 
comprised offenders who started a term of 
supervised release or probation on or after 
October 1, 2005. The subsequent case plan 
group comprised 193,586 case plan periods. 


Analysis 


A fairly straightforward and _ traditional 
approach was used in the development of 
the PCRA. Multivariate logistic regression 
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models'! were used to determine which items 
were superfluous. As a result, the total num- 
ber of items included in the multivariate 
model was reduced to ensure that statistical 
significance and direction of the relation- 
ship were maintained. Once the multivariate 
model was finalized, bivariate cross tabula- 
tions were used to assign appropriate weights. 
This method was chosen due to its trans- 
parency and, to date, there is little research 
indicating the superiority of complex weight- 
ing structures over dichotomous coding risk 
factors (see Gottfredson & Gottfredson, 1979; 
Silver, Smith, and Banks, 2000; Gottfredson & 
Snyder, 2005; Harcourt, 2007).'* The bivariate 
cross tabulations are presented in Appendices 
2-4. 

Once the final scoring algorithm was 
determined, a score was calculated with a cut- 
off score developed by visual inspection of the 
data. Although the data cutoffs were fairly evi- 
dent in the data, alternate cutoffs were tested 
with confirmation of best fit as determined 
through the use of chi-square statistics. A final 
set of analyses was conducted to determine 
how changes or stability in risk category from 
the beginning to the end of supervision was 
correlated with change in the probability of a 
new arrest. 


Findings 

Table 1 displays the results of a multivariate 
model predicting arrest during the initial 
case plan period using a split sample from the 
construction sample. As Table 1 shows, many 
of the variables included in the multivariate 
model were statistically significant at the .001 


'! When the outcome variable is composed of only 
two values (e.g., arrest or no arrest), which is typical 
for risk classification in probation, logistic regres- 
sion is usually the best approach to use. The main 
advantage of logistic regression is that few statistical 
assumptions are required for its use. In addition, 
it generates probability values that are constrained 
between zero and one. Logistic regression calculates 
the probability of an event occurring or not occur- 
ring (e.g., getting arrested or not getting arrested) 
and presents the results in the form of an odds ratio 
(Exp(B)). For the purposes of this article, the odds 
ratio is the number by which you multiply the odds 
of getting re-arrested for each one-unit increase 
in the independent variable (i.e., a variable in the 
equation). An odds ratio greater than 1 indicates 
that the odds of getting re-arrested increase when 
the independent variable increases; an odds ratio 
less than 1 indicates that the odds of getting re- 
arrested decrease when the independent variable 
increases (Menard, 2002). 


'? While the iterative classification processes seem 
to rate higher on some measures of utility, they also 
tend to have higher degrees of predictive shrinkage 
(see Silver et al., 2000). 


level. Odds ratios in the model also appear 
to be consistent with existing research that 
support well-accepted beliefs that alcohol and 
drug problems, unemployment, poor attitude 
(not motivated to change), criminal history, 
and lack of social support increase an offend- 
ers chances of getting re-arrested. Females 
appear to have a decreasing effect on the likeli- 
hood of re-arrest, which is also consistent with 
much of the existing research on gender and 
crime (Gendreau et al., 1996). 

From the multivariate analysis, variables 
were selected for inclusion on the risk assess- 
ment instrument (see Appendix 5). To gain 
a better understanding of the bivariate rela- 
tionships between the significant predictors 
in the multivariate model, we conducted 
a series of cross tabulations. Those results 
are reported in Appendices 2-4. In general, 
the bivariate cross-tabulations allowed us to 
assign 1 or 2 points to each of the factors. 
Although this approach may seem counter 
to prevailing wisdom on the development of 
weights for risk assessment, there is evidence 
that suggests that this approach produces 
an instrument that still outperforms clinical 
approaches to prediction (Dawes, 1979) and is 
more robust across time and sample variations 
(Gottfredson and Snyder, 2005; and McEwan, 
Mullen, & MacKenzie, 2009). 

Table 2 presents the descriptive statistics on 
the risk assessment score, which can theoreti- 
cally range from 0 to 19. There are 15 scored 
items. The scoring for each of the 15 items 
is displayed in detail in Appendix 5. Table 2 
presents the number of cases in each sample, 
minimum and maximum values, mean, and 


standard deviation of the linear risk score. 
There are no significant differences in the 
length of the prediction period or average 
risk score for the construction sample and 
first validation sample (6.46 and 6.43, respec- 
tively). However, there are differences in the 
mean risk score between the subsequent case 
plan sample and construction sample and sub- 
sequent case plan sample and first validation 
sample. The difference in prediction periods is 
a matter of policy, as the first case plan period 
is approximately 6 months while the third case 
plan is completed 12 months after the second 
case plan or 18 months after the beginning of 
supervision. The lower mean risk score might 
simply be a function of lower-risk offenders 
surviving supervision to the third and sub- 
sequent case plan periods. At any rate, there 
could be some debate that the difference in 
risk scores is not practically significant, and 
this argument might be valid since all three 
mean scores fall into the low-risk category. 
Table 3 presents the distribution of risk cat- 
egories by the type of sample used. In all three 
samples, low and low-moderate risk offend- 
ers accounted for at least 85 percent of the 
cases, whereas high-risk offenders accounted 
for only 1 percent. There was no statistically 
significant difference between the construc- 
tion sample and the validation sample at an 
alpha level of .01. However, there was a signifi- 
cant difference between the second validation 
sample (subsequent case plan) and the con- 
struction sample as well as between the second 
validation and the first validation sample. This 
is likely due to higher-risk offenders having a 
greater likelihood of revocation and thereby 


TABLE 2. 

Descriptive Statistics 
Sample Group N Minimum | Maximum Mean Std. Deviation 
Construction 51,428 6) 16 6.4634 2.83052 
Validation 51,643 6) 16 6.4272 2.80699 
Subsequent Case Plan | 193,586 ) 17 6.0320 2.73192 

TABLE 3. 

Distribution Across Risk Categories 

Sample 
Construction Validation Subsequent 
Case Plan 

Risk Category N % N % N % 
Low 19,080 37% 19,175 37% 83,037 43% 
Low-Moderate 24,751 48% ZONTS 49% 90,003 47% 
Moderate 7,019 14% 6,748 13% 19,244 10% 
High 578 1% 545 1% 1,302 1% 
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failing to survive to the second and subse- 
quent case plan periods. This finding, like that 
of the linear risk score, might be more an issue 
of sample size rather than holding practical 
significance. The change in the percentage 
of low-risk cases seems to be what drives the 
overall significant chi-square test. 

The next set of analyses focused on assess- 
ing the PCRAs predictive ability. AUC-ROC 
(Area of the Curve-Receiver Operating 
Characteristics)'* was chosen as the measure 
to assess prediction in large part because it 
is not impacted by base rates. Another con- 
venient property of the AUC-ROC over a 
correlation coefficient is that AUC-ROC is a 
singular measure and does not have differing 
calculations depending on level of measure- 
ment of the variables being evaluated (Rice & 
Harris, 2005). Table 4 displays the AUC-ROC 
between risk scores and re-arrests. A fourth 
sample (long-term follow-up) that includes 
initial case plan data on all offenders placed 
on supervision between September 30, 2005 
and September 30, 2006 is introduced in Table 
4. The data therefore allow for a follow-up 
period between three and four years. As Table 
4 shows, the AUC for each of the four sample 
groups is close to or exceeds the AUC-ROC 
value associated with large effect sizes (Rice & 
Harris, 2005). The AUC for the second valida- 
tion sample rose to .73, while the AUC for the 
long-term follow-up sample rose even higher 
to .78. Based on these results, the PCRA 
appears to have very good predictive validity 
in terms of accurately classifying offenders’ 
risk level. 

To put the AUC values into practical 
terms,'* we calculated the failure’ rates by 
each category of risk for each sample. These 
results are presented in Table 5. With the 
exception of the long-term follow-up sample, 


i3 


The AUC measures the probability that a score 
drawn at random from one sample or population 
(e.g., offenders with a re-arrest) is higher than that 
drawn at random from a second sample or popula- 
tion (e.g., offenders with no re-arrest). The AUC 
can range from .0 to 1.0 with .5 representing the 
value associated with chance prediction. Values 
equal to or greater than .70 are considered good. 
‘Rice and Harris indicate that the AUC holds the 
same meaning as the common language effect size 
indicator. That is, the probability that the PCRA 
score for a randomly selected recidivist is higher 
than the PCRA score for a randomly selected 
non-recidivist. For example, using the long-term 
follow-up data (AUC = .78), if you randomly select 
a recidivist and a non-recidivist, the recidivist’s 
PCRA score should be higher than the non-recidi- 
vist’s score 78 percent of the time. 

'’ Failure is defined as any new arrest during a 
term of supervision. 


the failure rates were relatively unchanged for 
a risk category across samples. For example, 
low-moderate risk offenders failed at a rate 
of 13 percent in both the construction and 
initial validation samples, and at 12 percent 
in the subsequent case plan sample. However, 
in the long-term follow-up sample, the low- 
moderate risk group’ failure rate increased 
significantly to 42 percent. Overall, the failure 
rate for the long-term follow-up group was 44 
percent, but the failure rate was significantly 
higher for high-risk offenders in this same 
group. Moderate-risk offenders failed at a rate 
of 71 percent and high-risk offenders had an 
83 percent failure rate. The uniform increase 
in failure rates across categories of risk and 
across the various samples continues to sup- 
port the validity of the PCRA. 

Survival analysis was conducted for each 
risk category and the survival curves associ- 
ated with those analyses are displayed in 
Figure 1. All possible data points, regardless 
of follow-up time, were used in the analysis.'° 
The follow-up period ranged from 0 to 60 
months. Survival rates for each risk category 


16 


STATA adjusts for cases that were lost during 
follow-up when calculating survival tables. 


are displayed at 6 months, 12 months, 36 
months, and 60 months. As Figure 1 shows, 
high-risk offenders have a very steep decrease 
in survival, as only 69 percent survived the 
first 6 months of supervision. As time passes, 
survival rates continue to drop rapidly for 
high-risk offenders, as only 46 percent sur- 
vived at 12 months and only 17 percent at 
36 months. After 60 months of supervision, 
a mere 6 percent of the high-risk offenders 
remain. In contrast to high-risk offenders, 
low-risk offenders have a significantly differ- 
ent experience on supervision. For example, 
while the survival rate for high-risk offenders 
was only 17 percent at 36 months, 90 percent 
of the low-risk offenders survived at this time 
period. Moreover, the survival rate for low- 
risk offenders decreased only 5 percentage 
points through 60 months to 85 percent. 
Low-moderate risk offenders have a sur- 
vival curve that is almost precisely between 
the survival curves of the low- and moderate- 
risk cases. Interestingly, the survival curve for 
the moderate-risk offenders seems to follow a 
form that is closer to the high-risk offenders 
than to the lower-risk offenders. Note that the 
survival rates continue to grow throughout 
the follow-up period for each group, and each 


TABLE 4. 

AUC-ROC Between Risk Score and Re-arrest’ 
Sample AUC Lower 95% Cl Upper 95% Cl | Significance 
Construction .709 .699 .000 
Validation .702 000 
Subsequent Case Plan 734 729 739 
Long-term Follow Up .783 778 .789 .000 


* Analyses based on TSR versus probation supervision were estimated. AUC-ROC values for the probation 
sub-sample were .65 (construction), .64 (validation), .72 (subsequent case plan), and .76 (long-term follow- 
up). While AUC-ROC values for the construction and validation samples were somewhat smaller than 
those generated for the overall sample, the AUC-ROC values for the subsequent case plan and long-term 
follow-up probation sub-samples were very similar to those generated for the overall sample. 


TABLE 5. 
Cross-tabulation between Risk Categories and Re-arrest 
Sample 

Risk Category Construction Validation psy ae Follow-Up* 
Low 5% 5% 4% 11% 
Low-Moderate 13% 13% 12% 42% 
Moderate 27% 28% 27% 71% 
High 39% 42% 41% 83% 
xX 1354.76 1444.74 6761.77 4997.40 


*Outcome measure is arrest for new criminal behavior only. 
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curve (with the exception of low-risk offend- 
ers) shows little sign of leveling off. 

One of the major benefits of third- and 
fourth-generation risk assessment is the abil- 
ity to measure change in risk over time. While 
many of the risk factors on the PCRA would 
be considered stable, some would also be 
considered acute (for a full discussion see 
Serin, Lloyd, & Hanby, 2010; Serin, Mailloux, 
& Wilson, 2010). Therefore, analyses were 
conducted that compared actual failure rates 
based on changes in initial and subsequent 
PCRA assessments. Table 6 outlines changes 
in failure rates based on first and last case 
plan assessment categories. The failure rates 
are based on the risk category for the last 
case plan period of the offender's supervision 
term; therefore, to be included in this table, 
the offender had to have at least two case 
plan periods that allowed for the scoring of 
the PCRA. According to the results presented 


in Table 6, not surprisingly, offenders in the 
higher risk categories (moderate and high) 
failed at a higher rate than offenders in the 
lower risk categories (low and low-moder- 
ate). However, offenders whose risk rating 
increased while under supervision appear 
to fail at a higher rate than offenders who 
maintained their initial rating through to their 
last assessment. For example, low-moderate 
risk offenders whose risk category increased 
to moderate had a failure rate of 41 percent, 
whereas low-moderate risk offenders who 
remained low-moderate risk or were reas- 
sessed as low risk had a failure rate between 
16 and 18 percent. Similarly, moderate-risk 
offenders who continued to be moderate risk 
had a 38 percent failure rate, while those who 
were reassessed as low-moderate had an 18 
percent failure rate and moderate-risk offend- 
ers reassessed as high risk had a 61 percent 
failure rate. 


FIGURE 1. 


Survival Analysis for the Four Risk Categories 
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TABLE 6. 
Changes in Failure Rates Based on First and Last Case Plan Assessment Categories 
Last Case Plan Assessment Category 
Initial Case Plan Assessment Category Low Low-Moderate | Moderate | High 
Low (n = 13,589) 4% 18% 
Low-Moderate (n = 15,660) 5% 16% 41% ~ 
Moderate (n = 3,581) = 18% 38% 61% 
High (n = 233) _ = 37% 53% 
x 237.65 396.23 162.85 10.54 


Discussion 


As previously stated, the purpose of this article 
is twofold: (1) To present the methodology 
and results produced in the development of 
the PCRA and (2) to discuss limitations of the 
PCRA as well as future developments. This 
article has provided details on the methods, 
measures, and sample used in the develop- 
ment of the PCRA. A fairly traditional model 
was followed in the development of the PCRA. 
Our efforts were supported by a relatively 
large dataset and fairly complete data. The 
sample was fairly representative of the popula- 
tion served and allowed for a construction and 
two validation samples. The overall results 
have demonstrated that the PCRA provides 
adequate predictive validity both in the short 
term (6-12 months) as well as in longer 
follow-up periods (up to 48 months). 

Multivariate analysis (see Table 1) of pro- 
posed predictors revealed that 15 factors 
were significantly related to the outcome of 
interest (new arrest). Seven additional factors 
tested were determined to be unrelated to a 
prediction of new arrest once the effects of the 
other factors were controlled. One additional 
measure, being female, was found to be sig- 
nificantly related to a new arrest. Subsequent 
models, not reported here, indicated that 
the addition of gender to the models yielded 
no increase in the predictive validity of the 
model. In addition, non-significant differ- 
ences were noted in the AUCs between males 
and females for each sample (i.e., construc- 
tion, validation, subsequent case plan, and 
long-term follow-up). Therefore, we con- 
cluded that the instrument performs equally 
well for males and females, even though the 
failure rates for males might be slightly higher 
than for females with similar risk scores. 

The creation of the risk score and cat- 
egories allowed for the identification of four 
risk categories: low, low-moderate, moderate, 
and high. Approximately 80 percent of each 
sample was made up of low and low-moderate 
risk offenders. Much smaller percentages were 
identified in each sample as moderate and 
high risk (approximately 12 percent and 1 
percent, respectively). Due to the distribution 
of risk categories being heavily skewed toward 
lower risk, the validity of the instrument may 
be brought into question. However, it should 
be noted that a current validated risk predic- 
tion instrument used in the federal system 
(RPI) yields a similarly skewed distribution. 
Analysis of failure rates by risk score and 
category using the PCRA yielded AUC-ROC 
values over the traditionally accepted value of 
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.70 and an AUC value for the long-term fol- 
low-up over .78. All of the AUC-ROC values 
were close to or exceeding the value associated 
with large effect sizes. Practically speaking, the 
instrument provided categorizations that are 
associated with the group failure rates that are 
differentiated and meaningful for meeting the 
risk principle (see Tables 4 and 5). 

The final analysis conducted in this study 
related to the dynamic nature of the PCRA. 
Recall from Table 6 that changes in actual fail- 
ure rates were associated with changes in risk 
category from the initial assessment to the last 
assessment. This finding is rather important, as 
it provides the opportunity to track meaning- 
ful changes in risk that occur throughout the 
supervision process. Moreover, Table 6 con- 
firmed that the PCRA identifies and measures 
dynamic risk factors that, apparently, when 
changed through supervision, services, or some 
other unmeasured process (natural desistance), 
lead to commensurate reductions in actual 
failure rates. The dynamic nature of the PCRA 
adds to its usefulness in developing case plans 
throughout the life of the supervision term. 


Limitations and Future Research 


Although this study was fairly comprehensive 
in scope and the dataset used was large and 
representative of the population served, there 
are a number of limitations and areas for 
future research that deserve mention. First, 
while the dataset was large and comprehen- 
sive, we have not investigated how scoring 
algorithms might be adjusted for each district. 
As with any measure, there is a distribution 
of AUC values when that test is calculated for 


each district. Data from 17 districts generated | 


AUC values below .70; however, only three 
districts had 95 percent confidence intervals 
that failed to cross the .70 threshold. While 
this finding may have been due to small 
samples in some districts, subsequent analysis 
should focus on bringing AUC values between 
risk scores and re-arrests up to larger values. 

A second limitation is that the data used 
in this research came from an administrative 
dataset. While it proved useful for our initial 
task of creating and validating a risk assessment 
instrument, it will be important to conduct 
similar validation analyses once we have an 
ample sample of offenders that were actually 
assessed using the assessment protocol. 

The third limitation involves the nature 
of the outcome measure being predicted. In 


this research we focused exclusively on the 


likelihood of a re-arrest and not the severity 
of the offense. We found it important to assess 
and determine the likelihoods of re-arrest as 
a first step in the assessment process. Because 
we do recognize that there is more than one 
dimension to an assessment in the criminal 
justice system, future analysis will focus on 
predicting the dangerousness of an offender. 

Fourth, while the PCRA is apparently 
dynamic, with changes in risk associated with 
changes in actual failure rates, it may not be 
sensitive enough for use on a monthly or 
shorter schedule. Due to the high value associ- 
ated with a dynamic risk assessment, it will be 
necessary to make the PCRA more sensitive to 
change, or supplement it with a more sensitive 
trailer assessment that increases its utility as a 
guide to service allocation. 

Finally, because rated but not scored items 
outnumber scored items on the assessment, 
future analysis will review the impact of rated 
but not scored items. For example, the PCRA 
currently has only one scored item in the area 
of cognitions. As a result of current testing 
on 80 self-report items that relate to criminal 
thinking styles, the number of scored items 
in the area of cognitions will likely increase. 


Continued analyses on rated but not scored 


items will also increase the understanding of 


the impact of self-reported attitudes, as well as 
guide adjustments to algorithms based on dis- 
trict, gender, and race differences, if relevant. 


Policy Implications 


Notwithstanding the limitations discussed 
above, there are two major policy implica- 
tions that stem from this research. First, the 
federal probation system now has a dynamic 
fourth-generation risk assessment for use on 
offenders under its jurisdiction. The instru- 
ment can be used to identify higher-risk 
offenders for enhanced services (see Andrews 
et al., 1990) and can also be used to identify 
targets for change to be addressed by external 
service providers. The second major policy 
implication is the apparent necessity for ongo- 
ing reassessment. Data analyzed in this study 
indicate that changes in levels of risk are 
associated with changes in actual failure rates. 
Therefore officers need to monitor risk in a 
standardized way to ensure that supervision 


and services are having intended impacts. If 


intended impacts are not being achieved, then 
officers will be able to modify supervision ser- 
vices to reduce the risk of recidivism. 
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APPENDIX 


APPENDIX 1. 
Rated Test Items 


Domain Factor Rating 
Criminal History Arrested Under Age 18 Yes/No 
Employment Number Of Jobs in Past 12 Months None/One/More than One 
Employment Employed Less than 50% of the Last 24 Months Yes/No j 
Substance Abuse Disruption at Work, Home, School Yes/No & 
Substance Abuse Use When Physically Hazardous Yes/No % 
Substance Abuse Legal Problems Related to Use Yes/No 5 
Substance Abuse Continued Use Despite Social/Interpersonal 

Problems Yes/No 
Social Networks Lives with Spouse and/or Children Yes/No 
Social Networks Lack of Family Support Yes/No 
Social Networks Companions Good Support and Influence/Occasional 


Association with Negative Peers/More Than 
Occasional Association with Negative 
Peers/No Friends 


Attitudes Antisocial Attitudes Yes/No 
Attitudes General Criminal Thinking (PICTS) Scale Scores 
Other No or Unstable Home One Address in Past 12 Months/More Than 


One Address in Past 12 Months or No 
Permanent Address 


Other Risk Influence at Home No Criminal Risks Present/Criminal Risks at Home 


Other Financial Stressors Adequate Income to Manage Debts/No Plan 
in Place to Meet Financial Debts, Expenses 
Exceed Income 


Other Pro Social Recreation Engages in Prosocial Activities/Has No Interests, 
Does Not Engage in Them, or Recreation Presents 
Criminal Risk 


Responsivity Low Intelligence Check Box 
Responsivity Physical Handicap Check Box 
Responsivity Reading and Writing Limitations Check Box 
Responsivity Mental Health Issues Check Box 
Responsivity No Desire to Change/Participate in Programs Check Box 
Responsivity Homeless Check Box 
Responsivity Transportation Check Box 
Responsivity Child Care Check Box 
Responsivity Language Check Box 
Responsivity Ethnic or Cultural Check Box 
Responsivity History of Abuse or Neglect Check Box 
Responsivity Interpersonal Anxiety Check Box 


Responsivity Social Security Card, Driver’s License, ID Check Box 
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APPENDIX 2. 


Cross Tabulations between Risk Factors and Re-arrest for Construction Sample 


Domain Variable Arrest Rate x2 P 
Criminal History Prior Arrests 
0 = No prior arrests 9% 
| = 1-2 prior arrests 12% 618.33 .000 
2 = 3-6 prior arrests 13% 
3 = 7 or more prior arrests 20% 
Criminal History Community Supervision Violations 
0 = No prior CS violations 11% 423.49 .000 
1 = 1 or more CS violations 20% 
Criminal History Varied Offending Pattern 
0 = 1 type of offending 14% 209.81 O00 
1 = 2 or more types of offending 20% 
| Criminal History Institutional Adjustment 
0 = No adjustment problems 12% 98.57 000 
| = Adjustment problems 22% 
Criminal History Violent Offense 
0 = No history or current violence 15% 50.405 .OOO0 
1 = History or current violence 19% 
Criminal History Age 
O= 41+ 11% 
638.77 
1 = 26-40 16% 
2 = 25 or younger 23% 
Education & Employment Highest Grade 
0 = High school degree or more 11% 467.44 000 
1 = GED or less than HS degree 18% 
| Education & Employment Unemployed 
0 = Currently employed 11% 318.08 000 
| = Currently unemployed 18% 
Education & Employment Good Work History 
0 = Stable work history 8% 352.17 000 
1 = Unstable work history 15% 
Substance Abuse Alcohol Problems 
0 = No current problems 12% 264.62 000 
1 = Current problems 28% 
Substance Abuse Drug Problems 
0 = No problems 12% 836.48 .00O 
| = Current problems 29% 
Social Networks Family Problems 
0 = No problems 12% 21377 .000 
1 = Current problems 18% 
Social Networks Married 
0 = Married 10% 187.69 O00 
1 = Single 16% 
Social Networks Social Support 
361.23 
O = Social support present 9% .O00 
1 = No social support 15% 
Attitudes Motivated to Change 
0 = Offender motivated to change 8% 473.99 000 
1 = Offender resistant to supervision 16% 


Note: Number of cases ranges from 31, 773 to 48,470 depending on risk factor. 
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APPENDIX 3. 


Cross Tabulations between Risk Factors and Re-arrest for Validation Sample 


Domain Variable Arrest Rate x2 P 
Criminal History Prior Arrests 
0 = No prior arrests 9% 
1 = 1-2 prior arrests 11% 612.91 000 
2 = 3-6 prior arrests 14% 
3 = 7 or more prior arrests 20% 
Criminal History Community Supervision Violations 
0 = No prior CS violations 11% 369.56 .000 
= 1 or more CS violations 19% 
Criminal History Varied Offending Pattern 
0 = 1 type of offending 14% 196.50 .000 
1 = 2 or more types of offending 20% 
Criminal History Institutional Adjustment 
0 = No adjustment problems 12% 87.241 .000 
1 = Adjustment problems 21% 
Criminal History Violent Offense 
0 = No history or current violence 15% 59.047 .000 
1 = History or current violence 19% 
Criminal History Age 
O = 41+ 11% 
499.76 .000 
1 = 26-40 16% 
2 = 25 or younger 22% 
Education & Employment Highest Grade 
0 = High school degree or more 11% 502.72 .000 
1 = GED or less than HS degree 18% 
Education & Employment Unemployed 
O = Currently employed 11% 379.277 .000 
1 = Currently unemployed 18% 
Education & Employment Good Work History 
0 = Stable work history 8% 371.27 .000 
1 = Unstable work history 15% 
Substance Abuse Alcohol Problems 
0 = No current problems 12% 283.03 .000 
1 = Current problems 29% 
Substance Abuse Drug Problems 
0 = No problems 12% 701.78 .000 
1 = Current problems 28% 
Social Networks Family Problems 
0 = No problems 12% 197.87 .000 
1 = Current problems 18% 
Social Networks Married 
O = Married 11% 164.99 .000 
1 = Single 16% 
Social Networks Social Support 
0 = Social support present 9% 398.44 .000 
1 = No social support 15% 
Attitudes Motivated to Change 
0 = Offender motivated to change 8% 507.97 .000 
1 = Offender resistant to supervision 16% 


Note: Number of cases ranges from 31, 607 to 48,434 depending on risk factor. 
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APPENDIX 4. 


Cross Tabulations between Risk Factors and Re-arrest for Subsequent Case Plan Periods 


Domain Variable Arrest Rate x2 P 
Criminal History Prior Arrests 
0 = No prior arrests 6% 
1 = 1-2 prior arrests 8% 3567.58 .OOO0 
2 = 3-6 prior arrests 11% 
3 = 7 or more prior arrests 17% 
Criminal History Community Supervision Violations 
0 = No prior CS violations 10% 2946.37 .OO0O 
1 = 1 or more CS violations 19% 
Criminal History Varied Offending Pattern 
0 = 1 type of offending 11% 1679.04 000 
= 2 or more types of offending 18% 
Criminal History Institutional Adjustment 
0 = No adjustment problems 11% 631.19 000 
1 = Adjustment problems 21% 
Criminal History Violent Offense 
0 = No history or current violence 11% 304.23 .0OO 
1 = History or current violence 16% 
Criminal History Age 
0 = 41+ 8% 
3163.72 
1 = 26-40 13% 
2 = 25 or younger 19% 
Education & Employment Highest Grade 
0 = High school degree or more 8% 2509.84 .000 
1 = GED or less than HS degree 15% 
Education & Employment Unemployed 
0 = currently employed 9% 1235.60 .000 
1 = currently unemployed 15% 
Education & Employment Good Work History 
0 = Stable work history 6% 2083.60 .000 
1 = Unstable work history 12% 
Substance Abuse Alcohol Problems 
0 = No current problems 11% 1344.46 .000 
1 = Current problems 24% 
Substance Abuse Drug Problems 
0 = No problems 9% 5720.49 000 
1 = Current problems 27% 
Social Networks Family Problems 
0 = No problems 9% 1254.19 000 
1 = Current problems 15% 
Social Networks Married 
0 = Married 8% 1096.37 .000 
1 = Single 13% 
Social Networks Social Support 
O = Social support present 9% 744.26 .000 
1 = No social support 12% 
Attitudes Motivated to Change 
0 = Offender motivated to change 7%, 2039.84 000 
1 = Offender resistant to supervision 13% 


Note: Number of cases ranges from 152,241 to 236,866 depending on risk factor. 
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APPENDIX 5. 
Scored PCRA Data Items 


VARIABLE NAME 


VARIABLE DESCRIPTION 


SCORED ITEM 


Date of Birth 


Record offender’s data of birth in MM/DD/YY format. 


Captured in 1.7 


# Adult Conv 


Record the total number of adult convictions. 


Captured in 1.2 


# Other Arrests 


Record the total number of other arrests. 


Captured in 1.2 


# Violent Arrests 


Record the total number of prior arrests for violent crimes. 


Captured in 1.3 


GED, code the highest grade completed in school. GED does not 
equal 12. 


# DV Record the number of arrests for domestic violence. Captured in 1.3 
HXSONC History of sex offending offenses without contact. Captured in 1.3 
HXSOC History of sex offending with contact. Code Y for yes, N for no, , 
Captured in 1.3 
and U for unknown. 
HXSOSR History of sex offending statutory rape. Code Y for yes, N for no, Captuied wi 
and U for unknown. 
HXSOO History of other sex offending. Code Y for yes, N for no, and U ee 
Captured in 1.3 
for unknown. 
Varied How many different types of offenses has the offender engaged Capnned nit 
in (property, drug, sex, violent, order, other)? 
Inst Adj1 Record the number of times an offender was written up during 
: Captured in 1.6 
prior terms of incarceration. 
Inst Adj2 Record the number of times the offender was officially punished 
for institutional infractions. 
CS Vio During how many previous periods of supervision did the 
offender a) commit a new crime or b) have violations that were Captured in 1.5 
reported to the court or paroling authority? 
High Grade Record the highest grade the offender completed. If received a 


Captured in 2.1 


Employed PSR 


Was the offender employed at the time of the pre-sentence 
report? Code Y for yes, N for no, and U for unknown. 


Captured in 2.2 


Employed Arrest 


Was the offender employed at the time of the arrest? Code Y for 
yes, N for no, and U for unknown. 


Captured in 2.2 


Alc Current 


Does the offender have a current alcohol problem? Code Y for 
yes, N for no, and U for unknown. 


Captured in 3.5 


Drug Current 


Does the offender have a current drug problem? Code Y for yes, 
N for no, and U for unknown. 


Captured in 3.6 
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IN 2009 THE ADMINISTRATIVE Office 
of the U.S. Courts and the Office of Federal 
Detention Trustee (a Justice Department 
agency charged with administering and con- 
trolling the costs of pretrial detention in 
the federal system) published Pretrial Risk 
Assessment in the Federal Court, which rec- 
ommended that the federal pretrial services 
system develop and implement an actuar- 
ial risk assessment tool. Ever since then, 
the system has been moving towards that 
goal. Lowenkamp and Whetzel (2009) have 
already detailed the process followed and 
described the tool that was ultimately devel- 
oped, the Pretrial Risk Assessment (PTRA). 
As of August 2011 the tool had been fully 
implemented in almost all districts; therefore, 
it seems an appropriate time to assess the tool 
in light of the available pretrial risk assessment 
literature, determine how implementation 
has proceeded in the federal system to this 
point, and assess the ultimate impact, if any, 
of the tool on the federal pretrial services 
system. Perhaps the most important question 
is whether the tool has begun to affect officer 
recommendations for release/detention and/ 
or release rates in districts where it has been 
operational for a year or more. 


Pretrial Services Risk 
Assessment Literature and PTRA 


One of the few areas in which pretrial ser- 
vices research initially led the way before 
our counterparts in post-conviction was risk 
assessment, with devices utilized in several 


of the larger cities, including Washington, 
D.C. (Toborg, Yezer, Tseng, & Carpenter, 
June 1984) and New York (Ares, Rankin, 
& Sturz, 1963), long before post-conviction 
risk assessment devices had been introduced 
there. Unfortunately, while such risk assess- 
ment tools remained in use in those cities, 
they did not spread to other pretrial services 
agencies as rapidly as risk assessment tools did 
in post-conviction organizations. In addition, 
assessing risk for offenders differs significantly 
from assessing risk for defendants, making it 
impossible for post-conviction and pretrial 
services to share risk assessment devices. For 
example, pretrial services risk assessment 
devices focus on failure-to-appear, which is 
not a focus of post-conviction tools; and post- 
conviction risk assessment devices focus on 
long-term recidivism, something that has not 
historically been a primary concern of pretrial 
services. Therefore, at least theoretically, there 
is little crossover between the two disciplines 
in the area of risk assessment. 

While the literature does not show a lot of 
work on risk assessment in pretrial services, 
a comparison with the post-conviction risk 
assessment literature finds that this is the 
evidence-based practice area in pretrial ser- 
vices that has received the greatest research 
attention and there are some studies of excel- 
lent quality that cover a wide range of issues 
(e.g., Toborg, Yezer, Tseng & Carpenter, 1984; 
Goldkamp & Gottfredson, 1988; Goldkamp 
& Vilcica, 2009; Levin, 2006). For example, 
Toborg, Yezer, Tseng, and Carpenter provide 


Timothy P. Cadigan 
Supervisory Pretrial Services Administrator 
Christopher T. Lowenkamp 

Probation Administrator 
Office of Probation and Pretrial Services 
Administrative Office of the U.S. Courts 


an excellent discussion of selectivity bias. First, 
arrested defendants are detained; because 
of this detention, their propensity for pre- 
trial arrest and failure-to-appear cannot be 
observed. Research on the first form of bias 
is fairly common in the literature and is 
discussed in most research on pretrial ser- 
vices risk assessment initiatives. However, 
rarely seen (but discussed in Toborg et al.) 
is the second form of selectivity bias, which 
affects defendants released under different 
scenarios: some without any restriction and 
others released on various bond types or with 
various conditions that are based on indi- 
vidual characteristics (Toborg, Yezer, Tseng 
& Carpenter, 1984:102). Toborg, Yezer, Tseng 
and Carpenter have done an excellent job of 
addressing this very important issue, which 
needs further discussion in the literature. 
Typically, defendants who are released on 
supervision in the federal system are given a 
“laundry list” of conditions. In fact, beginning 
in 2009, fully 99 percent of federal defendants 
who were released had one or more condi- 
tions in place. There has been little to no 
research establishing that any one condition or 
any combination of conditions helps achieve 
the desired goals of appearing in court and 
not committing an offense while on pretrial 
release. In addition, the conditions frequently 
result in technical violations, which can cause 
the defendant to be returned to the court for 
additional hearings, which can result in the 
defendant’s detention or in modification/ 
addition to the conditions. If these conditions 
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cannot be demonstrated to increase the likeli- 
hood that the defendant will appear in court 
as required and/or reduce new offenses com- 
mitted by the defendant while on pretrial 
release, then the significant investment of 
pretrial release agencies and courts in these 
conditions and their enforcement is ineffec- 
tive and unwise. In fact, preliminary research 
in the federal system seems to indicate that 
these conditions do not increase positive out- 
comes; instead, such conditions may increase 
negative outcomes (VanNostrand & Keebler, 
2009:31-33). In addition, a recently com- 
pleted literature review that focused on release 
conditions and supervision in all pretrial ser- 
vices entities (federal, state, county, and local) 
concluded that the evidence for the utility of 
such conditions is at best weak and in many 
cases nonexistent. “It must be acknowledged 
that research in this area is very limited and 
that more is needed” (VanNostrand, Rose, & 
Weibrecht, 2010:34). Therefore, more refined 
testing of the use of pretrial release conditions 
seems warranted. 

Toborg et al’s research, which used a team- 
developed risk assessment tool, produced the 
following very interesting findings: use of the 
tool led to more defendants being released 
on less restrictive conditions and with no 
increase in failure-to-appear or re-arrest rates 
(Toborg et al., 1984:105); the risk prediction 
tool that Toborg, et al developed increased 
release rates by 12 percent with no appreciable 
increase in failure-to-appear or re-arrest rates 
(Toborg et al., 1984:58); and, finally, the tool 
is more accurate for appearance in court 
than for safety (Toborg et al., 1984:73). Risk 


tools, while tremendously useful in improv- . 


ing agency decision making and ultimately 
release recommendations, have limitations. 
For instance, they are good at identifying 
groups of defendants who present various 
risks, but they cannot be totally accurate at 
the individual level (Toborg et al., 1984:111). 
Therefore, agencies need to convey to line 
staff, as the federal system has done, that the 
tool should not be followed blindly; in addi- 
tion, they should permit officers leeway to 
override the tool after staffing with a supervi- 
sor or some similar methodology. 

Finally, Toborg et al reported one finding 
that has yet to be replicated: pretrial services 
supervision had no effect on controlling the 
risk of nonappearance (Toborg et al., 1984:73). 
Further research is needed either to disprove 
or strengthen this finding due to the potential 
significance it carries if confirmed in other 
jurisdictions with other risk assessment tools. 


Once the new tool is fully implemented in the 
federal system, this finding will be tested. 

Goldkamp and Gottfredson studied three 
urban jurisdictions and presented a seemingly 
simple conclusion: that judicial involvement 
is essential to the successful implementation 
of a risk assessment device (Goldkamp & 
Gottfredson, 1988:129). That simple con- 
clusion also emerges in Goldkamp’s most 
recent research on pretrial risk assessment 
(Goldkamp & Vilcica, 2009:129-30). A 
finding that spans 22 years, appears in mul- 
tiple jurisdictions, and derives from different 
research partners would seem to hold poten- 
tial as a strong replicable result. Goldkamp 
and Gottfredson identified some ways to con- 
tribute to successful implementation through 
ongoing training, assessment of the officer's 
use of the tool, and annual or bi-annual certi- 
fication of the officer's skills in using the tool. 
In addition, the Goldkamp and Gottfredson 
study confirmed the major findings of Toborg 
et als earlier research. Unfortunately, the 
federal pretrial services risk assessment was 
not developed with judicial input. The early 
data emerging from its implementation in 
the federal system suggests that the lack of 
judicial input has made the tool less effec- 
tive in increasing officer recommendations 
for release and achieving increased rates of 
defendants released in the federal system. 
However, as implementation has progressed, 
opportunities for judicial training and input 
have been identified and implemented. The 
input from judicial officers is likely to con- 
tinue and even grow, providing opportunity 
for further refinement, based on judicial 
input, in the future. 

One of the strengths of the Goldkamp and 
Vilcica research is that it squarely takes on one 
of the most enduring “urban legends” of pre- 
trial services risk assessment research. Most 
pretrial services agencies, including the fed- 
eral system, continue to capture data on and 
analyze the variable “community ties.” While 
some of the fascination with this variable 
stems from its identification as an important 
variable in the “granddaddy” of all pretrial 
services research, the original Vera project, 
the “community ties” variable likely endures 
because of its tremendous “face validity.” 
Its inclusion in the small number of long- 
standing important pretrial services variables 
is certainly not warranted by the research 
results of the last 20 years. However, most 
researchers merely ignore the variable of com- 
munity ties, since the analysis does not bear 
out its value (see, for example, Administrative 


Office of the United States Courts, 1979; 
VanNostrand, 2003; VanNostrand & Keebler, 
2009; Winterfield, Coggeshall, & Harrell, 
2003). Goldkamp and Vilcica take on the lack 
of value of community ties for pretrial risk 
assessment and ultimately remove it from its 
lofty perch. 

Goldkamp’s analysis of factors 

influencing judicial decisions at 

the pretrial release decision, how- 

ever, found that contrary to the 

intended effect of Vera-type infor- 

mation-based reform procedures 

community ties items did not play 

a significant role in shaping judges’ 

actual pretrial custody decisions- 

and were not helpful predictors 

of defendant risk. (Goldkamp & 

Vilcica, 2009: p. 124). 

In the research used to develop the PTRA, 
a “community ties” variable known as “Time 
in Area’ was tested and found to have no pre- 
dictive value in the PTRA model. 

A seemingly “obvious” issue not found 
in virtually any other research on the topic 
of risk assessment is the importance of 
including judicial officers in the develop- 
ment, implementation, and ongoing use of a 
risk assessment device. Only Goldkamp and 
Vilcica’s findings discuss this issue, not to 
mention endorsing the strong role it played 
in the Philadelphia research: “As a judicially 
developed and adopted policy, it stands alone 
in the nation in the first years of the 21st 
century-one might argue, in isolation—-as an 
empirically informed approach to the prob- 
lem of judicial discretion at the bail stage” 
(Goldkamp & Vilcica, 2009:129-30). As men- 
tioned earlier, the federal system is currently 
implementing a risk assessment tool without 
judicial involvement, which could be impact- 
ing the acceptance and use of the tool in the 
federal system, although opportunities for 
judicial training and input have progressively 
increased in the course of implementation. 

Given Goldkamp and Vilcica’s vision of 
pretrial justice and their desire to improve the 
pretrial release process and reduce judicial 
discretion, it is almost shocking that they 
missed the importance of pretrial deten- 
tion and made their tool detention neutral 
(Goldkamp & Vilcica, 2009:134). This is espe- 
cially true since Philadelphia has operated 
pretrial services under federal court supervi- 
sion due to jail overcrowding at various times 
during the 20-plus years encompassed by the 
guideline project in Philadelphia. Reducing 
unnecessary pretrial detention needs to be a 
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core principle for pretrial services and judicial 
officers, given the negative consequences of 
pretrial detention at subsequent phases of the 
criminal justice system. The negative impact 
of detention on defendants has previously 
been documented. 

Defendants incur significant costs when 
they are detained while their case is pend- 
ing (Goldkamp & Gottfredson,1985:17; 
Foote, 1954; Klein, 1997:292; Williams, 
2003; Rankin, 1964; Tarturo & Sedelmaier, 
2009:218). Research has consistently shown 
that a defendant's pretrial release or detention 
status impacts case disposition and ultimately 
sentence. Proponents of pretrial release argue 
that several factors contribute to that out- 
come: 1) detainees have reduced access to 
their attorneys, which limits the defendant's 
ability to fully participate in the preparation 
of the defense case; 2) detention exerts great 
pressure on the defendant to plea bargain the 
disposition of the case; and 3) detention cre- 
ates negative perceptions of the detainee in 
the minds of the court/jury who convict and/ 
or sentence the defendant. Goldkamp con- 
cluded that the effect of detention was more 
limited, in that its only demonstrated impact 
was an increased risk of receiving a sentence 
of incarceration when compared to released 
defendants. 

Levin's research revealed that a defendant's 
odds of failing to appear in a county that uses 
a quantitative risk assessment are .40 times 
lower than the odds faced by a defendant 
appearing in a county that uses qualitative 
risk assessment (Levin, 2006:10). In addition, 
if the county uses some mix of quantitative 
and qualitative measures, defendants are still 
less likely to fail to appear than if they used 
qualitative alone (Levin, 2006:10). Finally, if 
the county uses some mix of quantitative and 
qualitative measures, defendants are also less 
likely to be rearrested (Levin, 2006:11). 

The literature on pretrial services risk 
assessment clearly establishes several 
important premises: “objective risk assess- 
ment produces more non-cash release 


recommendations” (Cooprider, 2009:15); 
“Notwithstanding a broader definition of ‘pre- 
trial failure’ and cutting field contacts in half, 
violation rates declined or remained stable 
since the implementation of objective risk 
assessment” (Cooprider, 2009:15); and predic- 
tive items identified in pretrial services risk 
assessment research change over time and 
therefore must be re-validated on an ongoing 
basis to ensure their integrity and effective- 
ness (e.g., VanNostrand, 2003; VanNostrand 
& Keebler, 2009; Siddiqi, 2002). 

The need for validation of pretrial risk 
assessment tools is illustrated by the follow- 
ing example of an established risk assessment 
finding that is likely to change. Risk prediction 
research in the city of New York for the past 20 
years has established with relative consistency 
the predictive value of having a telephone 
in the residence of the defendant. Given the 
move in the past decade from the dominance 
of landline technology to increasing reliance 
on cell phone technoiogy, it seems unlikely 
that future research will continue to find 
great predictive value for a landline phone in 
the defendant's residence (Siddiqi, 2002:2). 
Fortunately for citizens in New York City, 
the agency providing pretrial services has an 
excellent research operation that re-validates 
its risk prediction tool every three to five years 
as warranted. Ongoing re-validation is an 
essential step for all pretrial risk assessments 
and it will be an ongoing component of the 
federal risk assessment program. 


Federal Risk Assessment 


One of the major recommendations of the 
Office of Federal Detention Trustee research 
is that the pretrial services system should 
develop and implement an actuarial risk 
assessment tool VanNostrand & Keebler, 
2009). The Pretrial Services Risk Assessment 
tool was constructed using the same data 
employed in the Office of Federal Detention 
Trustee research. The Office of Probation and 
Pretrial Services hired a staff person proficient 
in the development of actuarial devices and 


ultimately developed the tool internally. The 
result is an objective, actuarial instrument 
that provides a consistent and valid method 
of predicting risk of failure-to-appear, new 
criminal arrest, and technical violations that 
lead to revocation while on pretrial release. 
The instrument contains 11 scored and 9 
unscored items. The unscored items are rated 
as either A or B and do not contribute to the 
current overall risk score. However, they will 
be analyzed for future revisions aimed at 
improving the predictive value of the tool. The 
scored items are given a number of points (0, 
1, or 2), which are then added up to produce 
an overall score. When administered correctly, 
the Pretrial Services Risk Assessment provides 
a score that allows for classification into a 
risk category. Those risk categories are then 
associated with rates of failure-to-appear, new 
criminal arrest, and technical violations lead- 
ing to revocation (See Table A). 

When a defendant or material witness is 
arrested or summoned to appear before the 
court for an initial appearance, the magistrate 
judge typically requires a pretrial services 
report based on the investigation conducted 
by the pretrial services officer. The officer 
interviews the defendant to gather informa- 
tion for the report, which contains: defendant 
case information, including residence; family 
ties; employment history; financial resources; 
health (including mental health and substance 
abuse histories); and criminal history. Based 
on this information, the officer will provide 
the court with an assessment of whether or 
not the defendant is likely to appear for court 
proceedings in the future and whether he 
or she presents a danger to the community. 
Finally, the last section of the report provides 
the officer’s recommendation to the court for 
the release or detention of the defendant. Once 
an officer has been trained on the new risk 
prediction tool, that recommendation should 
routinely be based on the Pretrial Services 
Risk Assessment, although the officer can 
depart from the tool’s recommendation after 
staffing the results with his or her supervisor. 


TABLE A. 

Likelihood of outcomes based on event occurring during pretrial period. 
Risk Category N % Risk Score FTA NCA FTA/NCA TV FTA/NCA/TV 
Category 1 52,677 29 0-4 1% 1% 2% 1% 3% 
Category 2 52,653 29 5-6 3% 3% 5% 4% 9% 
Category 3 49,920 27 7-8 4% 5% 10% 9% 18% 
Category 4 21,779 12 9-10 6% 7% 15% 15% 28% 
Category 5 4,710 3 11+ 6% 10% 20% 19% 35% 
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Implementation 


Once developed, the Pretrial Services Risk 
Assessment tool was piloted in several dis- 
tricts and formal implementation of the tool 
began in January 2010. As of June 2011, there 
were 72 districts “live,” using the tool on “all” 
cases; 87 districts trained and certified in 
using the Pretrial Services Risk Assessment 
tool; and 6 districts yet to be trained.' In terms 
of numbers of officers, as of June 1,742 pretrial 
services officers had been trained and 1,338 
certified to use the tool effectively. National 
implementation was completed in almost all 
93 districts by August 2011. 

The certification process should also pro- 
vide a measure of the effectiveness of the 
tool and the quality of the training; however, 
all of the necessary data elements are not 
available in a format that allows them to be 
extracted and analyzed. The certification pro- 
cess requires the user to successfully complete 
two of three risk assessment tool scenarios. At 
this writing no officer has failed to be certi- 
fied through that process; however, data is not 
available on whether all officers passed after 
completing just the two scenarios. Were that 
the case, it would indicate successful training 
and the tool's effectiveness. While trainers can 
be encouraged that no trained officers have 
failed to be certified, conclusions beyond that 
are not warranted without the actual data. 

The pretrial risk assessment was first imple- 
mented in two pilot districts (Nebraska and 
North Carolina Western), where forms, poli- 
cies, and procedures were also implemented 
and tested. The pilot focused primarily on 
issues such as ease of use to insure that the 


national implementation went forward with 


as few complications as possible. That pilot 
led to such changes to the tool as clarification 
of definitions, modification of the tool form, 
etc. However, those changes were for the most 
part routine and implementation progressed 
without major problems. 

The only remaining concern in the imple- 
mentation is the pace of districts using the 
tool on all cases. The federal system aver- 
ages about 26,000 pretrial investigations and 
reports per quarter. Unfortunately, the PTRA 


' Pretrial services cases in the District of Columbia 
are not classified as federal pretrial services cases by 
the Pretrial Services Act of 1982; therefore, only 93 
of the 94 federal districts are included for pretrial 
services data. 


TABLE B. 
Detention Rates by Risk Category for the Last Five Years 
PTRA 1 PTRA 2 PTRA 3 PTRA 4 PTRA 5 
FYO6 16.7% 37.0% 55.2% 73.6% 86.9% 
FYO7 17.7% 37.3% 55.8% 73.1% 86.4% 
FYO8 19.4% 37.4% 54.8% 72.3% 85.0% 
FYO9 18.8% 39.6% 56.2% 70.4% 83.1% 
FY10 18.5% 38.6% 55.0% 69.0% 81.6% 
FY06-10 18.2% 38.0% 55.4% 71.8% 84.8% 


is averaging about 4,000 per quarter, leaving 
22,000 reports without PTRA scores. That 
pace must improve significantly and quickly. 
OPPS has begun tracking investigations/ 
reports and PTRA scores by district to enable 
us to quickly identify and address districts that 
are not producing a PTRA for every report 
submitted. 


Initial Results 


The implementation of the tool has generated 
great debate over the finding (represented in 
the scores of “1” for defendants charged with 
violent offenses) that violent defendants in 
fact perform better than most other defen- 
dants in terms of re-arrest, failure-to-appear, 
and technical violations leading to revoca- 
tion of pretrial release. However, the results 
found in the federal study are consistent with 
other similar findings: “defendants charged 
with more serious offenses and violent have 
not posed a high risk of failure pending trial 
in past research” (Austin, Krisberg & Litsky, 
1984:30; VanNostrand & Keebler, 2009:21; 
Toborg et al., 1984:56). Pretrial services offi- 
cers can be quick to convert long rap sheets 
into detention recommendations, citing pub- 
lic safety; however, in many cases the evidence 
does not support that conclusion. 

Early results from the first three months 
of implementation in the two pilot districts 
(Nebraska and North Carolina Western) 
showed that the tool increased officer recom- 
mendations in favor of release, which was a 
desired goal of the risk assessment tool. The 
early pilot results showed no impact from the 
tool on actual release rates. In the first year 
of operation, after excluding immigration 
cases (since the vast majority of those defen- 
dants are detained), for the January 1, 2010 
through December 31, 2010 timeframe the 


Western District of North Carolina increased 
recommendations for release 13.5 percent 
and increased defendants actually released 6.1 
percent. In the first year of operation, again 
excluding immigration cases and focusing 
on the one-year period from January 1, 2010, 
through December 31, 2010, the District 
of Nebraska increased recommendations for 
release 2.0 percent and increased defendants 
actually released 1.4 percent. These are sig- 
nificant trends for the pilot districts and we 
can hope that those trends will continue 
nationwide as use of the tool becomes more 
widespread and ultimately universal. 


Future Questions 


The largest issue facing the federal pretrial 
services system is unnecessary pretrial deten- 
tion. As Table B shows, significant percentages 
of low-risk defendants (PTRA 1 & PTRA 2) 
have been detained over the past five years and 
in all likelihood will continue to be detained 
unless two things occur: pretrial services 
officers prepare strong, factual, and accurate 
pretrial services reports that contain strong 
release recommendations and (where appro- 
priate) release packages designed to protect 
the community and ensure pretrial justice for 
the defendant, and 2) United States magistrate 
judges assume a higher level of risk in select- 
ing defendants for release than they been 
willing to assume in the past five years. For 
magistrate judges, such a change in practice 
would itself be risky, since they must apply the 
Bail Reform of 1986 to the more than 100,000 
defendants who appear before them each year. 
Such openness to pretrial release is not for the 
faint of heart. Luckily the position is staffed 
by some of the most talented and dedicated 
public servants to ever serve in the federal 
justice system. 


he 


34 FEDERAL PROBATION 


Volume 75 Number 2 


References 
Ares, C. E. Rankin, A. & Sturz, H. (1963). The 


Manhattan bail project. New York University 


Law Review, 38-1, p. 67-95. 

Austin, J., Krisberg, B., & Litsky, P. (1984). 
Evaluation of the field test of supervised 
pretrial release final report. Washington, 
D.C.: National Institute of Justice. 

Cadigan, T. P. (2008, September). Evidence- 
based practices in federal pretrial services. 
Federal Probation. 72-2, p. 87-90. 

Cadigan, T. P. (2009, September). Implementing 
evidence-based practices in federal pretrial 
services. Federal Probation, 73-3, p. 30-32. 

Cooprider, K. (2009, June). Pretrial risk assess- 
ment and case classification: A case study. 
Federal Probation, 73-1, p. 12-15. 

Foote, C. (1954). Compelling appearance 
in court: Administration of bail in 
Philadelphia. University of Pennsylvania 
Law Review, 102: pp. 1031-1079. 

Goldkamp, J. S., & Gottfredson, M. R. (1985). 
Policy guidelines for bail: An experiment 
in court reform. Philadelphia: Temple 
University Press. 

Goldkamp, J. S. & Gottfredson, M. R. (1988). 
Development of bail/pretrial release 
guidelines in Maricopa County Superior 
Court, Dade County Circuit Court and 
Boston Municipal Court. Washington, D.C.: 
National Institute of Justice. 

Goldkamp, J.S. & Vilcica, E.R. (2009). Judicial 
discretion and the unfinished agenda 
of American bail reform: Lessons from 
Philadelphia's evidence-based judicial 


strategy. Studies in law, politics, and society 
volume 47: New perspectives on crime and 
criminal justice. Emerald Publishing: Bingly, 
UK, pp. 115-157. 


Klein, D.J. (1997). The pretrial detention “crisis”: 


The causes and the cure. Journal of Urban 
and Contemporary Law, 52-28, pp. 281-305. 

Latessa, E. J. & Lovins, B. (2010) The role of 
offender risk assessment: A policy maker 
guide Victims and Offenders: Journal of 
Evidence-based Policies and practices. 
5(3):203-219. 

Levin, D. (2006, November). Examining the 
efficacy of pretrial release conditions, 
sanctions, and screening with the state court 
processing statistics dataseries. Washington, 
D.C.: Pretrial Justice Institute. 

Lowenkamp, C. T. & Whetzel, J. (2009, 
September). The development of an 
actuarial risk assessment instrument for 
US. Pretrial Services. Federal Probation, 
73-3, p. 33-36. 

Mamalian, C. A. (March 2011). State of 
the Science of Pretrial Risk Assessment. 
Washington, D.C.: Bureau of Justice 
Assistance. 

Rankin, A. (1964). The effect of pretrial 
detention. New York University Law Review, 
39, p. 641. 

Siddiqi, Q. (2005, October). Predicting the likeli- 
hood of pretrial rearrest: An examination of 
New York City defendants. New York: New 
York City Criminal Justice Agency. 

Siddiqi, Q. (2002, June). Prediction of pretrial 
failure to appear and an alternative pretrial 


release risk-classification scheme in New York 
City: A reassessment study. New York: New 
York City Criminal Justice Agency. 

Tarturo, C. & Sedelmaier, C.M. (2009, June). A 
tale of two counties: The impact of pretrial 
release, race, and ethnicity upon sentencing 
decision. Criminal Justice Studies, 22-2, pp. 
203-221. 

Toborg, M. A. Yezer, A. M. Tseng, P. & 
Carpenter, B. L. (1984, June). Pretrial release 
assessment of danger and flight: Method 
makes a difference. McLean, VA: Lazer 
Management Group. 

VanNostrand, M. (2003). Assessing risk 
among pretrial defendants in Virginia: The 
Virginia pretrial risk assessment instrument. 
Richmond, VA: Virginia Department of 
Criminal Justice Services. 

VanNostrand, M. & Keebler, G. (2009, April). 
Pretrial risk assessment in the federal 
court. Washington, D.C.: Office of Federal 
Detention Trustee. 

VanNostrand, M., Rose, K.J. & Weibrecht, K. 
(December 2010). In pursuit of legal and 
evidence-based pretrial release recommenda- 
tions and supervision. Richmond, VA: 
Virginia Department of Criminal Justice 
Services. 

Williams, M.R. (2003). The effect of pretrial 
detention on imprisonment decisions. 
Criminal Justice Review, 28-2, pp. 299-316. 

Winterfield, L., Coggeshall, M., & Harrell, A. 
(2003, April). Development of an empirically- 
based risk assessment instrument: Final 
Report. Washington, D.C.: Urban Institute. 


September 2011 


THE OFFICE OF PROBATION and Pretrial 
Services (OPPS) of the Administrative Office 
of the U.S. Courts (AO), in keeping with guid- 
ance from the Criminal Law Committee of 
the Judicial Conference, has worked to sup- 
port and guide the adoption of evidence-based 
practices in the probation and pretrial services 
system. The adoption of an evidence-based 
philosophy requires the use of assessment 
tools; the development of supervision and 
programming options that are informed by 
evidence; the use of research; and the develop- 
ment of staff skills to support programming 
and service delivery. The supervision and pro- 
gramming principles derived from research, 
when implemented well, lead to a more effi- 
cient and effective system, maximizing the 
reduction in recidivism. 

In August 2009, OPPS formed the 
Evidence-Based Practice (EBP) Working 
Group to assist OPPS in developing a plan 
for the strategic implementation of evidence- 
based principles in the federal probation and 
pretrial services system. The working group 
embodies a team approach, with members 
representing the probation and pretrial ser- 
vices officers from various districts, OPPS, 
and the Federal Judicial Center (FJC). 

The working group first met for three days 
in October 2009. Before the meeting, group 
members were asked to review literature and 
research that would be used to help guide our 
conversations in a way that would maximize 
the value of our time together. Our initial 
meeting included discussion and lively debate 
on various topics, including but not limited to: 
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a historical context of the movement toward 
evidence-based practice in the federal sys- 
tem; the principles of risk, need, responsivity, 
fidelity and measurement; use of an actuarial 
risk assessment instrument; supervision skills; 
case planning; and implementation. Our 
review of the literature focused on: (1) our 
general impressions; (2) information gleaned 
from the various articles; and (3) ways to make 
interaction with defendants and offenders 
purposeful, intentional, and productive. 
Throughout the course of the meeting, 
several things became evident and ultimately 
led us to identify three primary goals: 
® Goal 1: Educate all and provide a com- 
mon foundation and understanding of EBP. 
Unlike a local, county, or state system 
where things may be easier to implement 
uniformly and/or control, we are 94 federal 
districts with our own unique issues and 
cultures. We have varying levels of interest 
and varying levels of readiness. Due to the 
varying levels of readiness, we identified 
the need to establish a common foundation 
of knowledge as a primary goal. 
© Goal 2: Implementing an Actuarial Risk 
Assessment. Based on the research, the 
working group concluded that the use of 
an actuarial risk/needs assessment was 
absolutely necessary, and needed to be the 
foundation to build upon. After much dis- 
cussion, the working group unanimously 
endorsed the use of the Post Conviction 
Risk Assessment (PCRA) for a variety of 
reasons including: 1) it was developed 
using data from our federal population 2) 


it provided continuity among the districts; 

and 3) unlike an off-the-shelf item, it was 

specifically designed to fit our needs. 

® Goal 3: Teaching officers supervision skills 
that will have the greatest effect on reduc- 
ing recidivism. Research tells us that the 
skills used by officers during their work- 
ing alliance with offenders are a critical 
component to lasting change. The working 
group felt that it is essential for officers to 
have this skill set. 

The working group also believed that 
transparency would be important as the sys- 
tem moved toward becoming outcome based. 
We thought that openness and inclusion of 
all federal probation and pretrial services 
districts was a critical component. In order 
to extend as much information as possible 
beyond the working group, we agreed to send 
out our information in the weekly memos 
that are issued from the Office of Probation 
and Pretrial Services to district chiefs as well 
as through other avenues available to us, 
such as News e& Views, the internal newslet- 
ter for federal probation and pretrial services; 
Chiefs Conferences; the American Probation 
and Parole Association (APPA); Chief and 
Deputy Chief Administrative Meetings; 
Chiefs Advisory Group meetings; various 
trainings and conferences; and our system's 
EBP website. 

Recognizing the enormity of our mission 
and the system-altering shift we were sug- 
gesting, the working group concluded it was 
necessary to put forth a document that pro- 
vided chiefs and districts with a “Blue Print” to 
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incorporate evidence-based practices in their 
districts, with the goal of changing offender 
behavior and reducing recidivism. We also 
believed it was important to hold a sympo- 
sium for all chiefs to bring everyone together 
to discuss this important topic. Shortly after 
our initial meeting concluded, the Chiefs 
Advisory Group was given a summary of this 
meeting and advised of our plan to create a 
“Blue Print” and to plan a Chiefs Symposium. 
The Chiefs Advisory Group was very support- 
ive, and endorsed the idea of the symposium. 

As co-chairs of the working group, we 
embarked on creating a draft “Blue Print” 
that was based upon the relevant research and 
encompassed all of the information the work- 
ing group discussed, reviewed, and debated 
during our meetings. The National Institute 
of Corrections (NIC), in concert with the 
Crime & Justice Institute (CJI), developed An 
Integrated Model for the effective implementa- 
tion of evidence-based practices that combines 
Evidence-based Principles; Organizational 
Development; and Collaboration. The “Blue 
Print” embraces this integrated model with an 
approach that takes into account the different 
levels of readiness in the federal probation and 
pretrial services districts. The plan supports 
both interdependence and local innovation. 
The eight principles for effective interven- 
tions outlined by the NIC/CJI have been used 
as a starting point and foundation for the 
federal system to build upon. The strategic 
plan utilizes the conceptual framework of the 
eight, while incorporating others that have 
been supported by empirical data to identify 
five core principles of effective intervention 
for our system. Initiating and maintaining 
offender interventions and supervision prac- 
tices consistent with these five principles will 
maximize our system's ability to reduce recidi- 
vism rates. The five principles are: Risk, Need, 
Responsivity, Fidelity, and Measurement. 

Our initial draft of the blueprint was vetted 
through a subcommittee of the working group 
before eventually being vetted through the 
entire working group. On December 11, 2009, 
the working group approved a draft document 
that was posted on the federal judiciary’s inter- 
nal website (J-Net) for comments from the 


field. We then focused on the plan and agenda 
for the symposium. In February 2010, a two- 
day Chief's Symposium on Evidence-Based 
Practice took place in Houston, TX. 

Since the development of the blueprint and 
the Chief's Symposium, significant progress 
has been made toward achieving the goals 
identified in the strategic plan. 

® Goal 1: Ensure a common foundation of 
knowledge and understanding of EBP and 
research on risk and reducing recidivism 
by providing EBP 101 to all districts. 


Working collaboratively with OPPS and 
the Administrative Office of the U.S. Court's 
Office of Public Affairs, the working group 
helped develop EBP 101 training for the 
field. The training, which was held live in 
the District of Maine, was taped and has 
been available to all districts through the 
federal judiciary’s intranet since October 
2010. Districts conducting the training with 
the use of the viewer's guide have reported 
much greater success with training and imple- 
mentation of the new Post Conviction Risk 
Assessment (PCRA). Feedback suggests that 
districts find the practical discussion very 
useful in addition to the lecture portion that 
discusses the science behind the principles. 


®@ Goal 2: Ensure the national use of the Post- 
Conviction Risk Assessment by September 
2011 with the rollout of PACTS Gen 3. 


Although we have no control over the roll- 
out of PACTS Gen 3, the newest generation of 
our system's Probation and Pretrial Services 
Automated Case Tracking System, we have suc- 
cessfully completed 13 PCRA training sessions 
and as of this writing in the summer of 2011, 
have 3 remaining. By September 2011, all cur- 
rent supervision officers will have completed 
the training. Along the way, there have been 
minor adjustments to the training based on 
feedback received from attendees, information 
gleaned from the Pre-PCRA Conference Calls, 
and districts’ completion of EBP 101. Trainers 
have become increasingly comfortable with the 
material and process, and feedback from the 
training sessions has been positive. 


© Goal 3: Ensure that officers are providing 
effective interventions in their direct con- 
tact with offenders with Core Correctional 
Skill Building. 


Over the last year OPPS has trained 155 
officers in STARR (Strategic Techniques 
Aimed at Reducing Re-Arrest). As a system, 
we are beginning to build a contingency of 
STARR coaches, which is a critical part of the 
process. The EBP Working Group and OPPS 
are continuing their efforts to develop models 
of delivery for STARR training. Interested 
districts need to be able to support imple- 
mentation efforts by using the PCRA and 
developing local coaching expertise. 

In concert with the Office of Probation and 
Pretrial Services and the EBP Working Group, 
the Federal Judicial Center committed to take 
on the domains of organizational develop- 
ment and collaboration set out in the NIC 
Integrated Model, to provide education and 
consulting services to probation and pretrial 
managers, district and magistrate judge, and 
federal defenders. The following programs 
have been developed or reengineered to meet 
this goal: 

e Facilitating Offender Reentry to Reduce 
Recidivism: A Workshop for District 
Teams 

® Quality Improvement Seminar for Federal 
Reentry and Post-Conviction Drug Courts 

e Leading Organizational Transitions 
Training for Trainers 

@ Customized Consultation on EBP and 
Reentry Projects 
Executive Team Training 

@ New Chief Training 
New and Experienced Deputy Chief 
Training 

@ New and Experienced Supervisor Training 
As co-chairs of this very active working 

group, we are proud of the commitment, 

motivation, and dedication the members of 
our group have demonstrated. We continue to 
work diligently to meet our goals and to pro- 
vide the field with the most current research 
and information relevant to the incorporation 
of evidence-based practices in the United 
States Probation and Pretrial Services System. 
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IN THE FALL OF 2010, the Federal Judicial 
Center (Center) surveyed chief United States 
probation officers to gather information about 
the status of federal post-conviction supervi- 
sion programs that are modeled on state and 
local drug and reentry courts. The survey 
was undertaken at the request of the Judicial 
Conference Committee on Criminal Law as 
part of a larger study of reentry programs in 
the federal system.' This report summarizes 
the survey results in the areas of program 
goals, design, resources, and reach and dis- 
cusses the role and resource implications of 
various design features. 

The survey proceeded in two stages. On 
October 4, 2010, the 93 probation chiefs 
were polled on whether their districts had 
or were planning judge-involved supervision 
programs. Districts known from previous 
surveys or trainings to have programs were 
asked if the program was still operating and, 


' The Center, in cooperation with the 
Administrative Office of the U.S. Courts’ Office 
of Probation and Pretrial Services, is conducting 
a two-pronged study of the operational aspects, 
outcomes, and cost-effectiveness of judge-involved 
reentry programs, including an evaluation of their 
effectiveness in comparison to other offender super- 
vision approaches that may require less intensive 
use of judicial resources. In addition to the pro- 
cess-descriptive assessment of already-established 
programs of which this survey is a part, the study 
also includes a multi-year effort to implement an 
experimental design, with random assignment, in 
five districts with new or relatively new programs: 
California Central, Florida Middle, lowa Southern, 
New York Southern, and Wisconsin Eastern. 


if so, to confirm the start date.* Others were 
asked if they had or were planning a program 
described as follows: 
The types of programs in which the 
Committee is interested are those 
that employ the authority of the 
court to impose graduated sanctions 
and positive reinforcements in a less 
formal team approach to supervis- 
ing targeted offenders. The team 
involves a judge and typically rep- 
resentatives of the probation office, 
US. Attorney's office, and public 
defender or other defense bar repre- 
sentative. Service providers may also 
play a role as either members of or 
advisors to the team.* 

This poll identified 41 districts that were 
(or would be by end of year 2010) operating 
45 judge-involved supervision programs. On 
October 28, 2010, the Center sent a detailed 
survey to the chief United States probation 
officers in 36 districts that host 39 of these 
programs. (Courts participating in an experi- 
mental segment of the larger study of reentry 
programs‘ and programs in operation for less 
than six months were excluded.) All surveys 
were returned before the end of November. 


2 


The Center surveyed probation chiefs in 2008 
to see if they had programs modeled after drug or 
reentry courts, and both the Office of Probation 
and Pretrial Services and the Center's Education 
Division maintain program contact lists. 


3 


Although the survey was titled “2010 Reentry 
Program Survey of Chief Probation Officers,” it in 
fact included all judge-involved supervision pro- 
grams, regardless of the target population. 


* Supra Note 1. 


Barbara Meierhoefer 
Research Consultant 
Federal Judicial Center 


Program Development 
and Training 


Over three-quarters of the federal judge- 
involved supervision programs were 
developed at the request of the court. This is 
a relatively new movement, with the major- 
ity of programs starting within the last two 
years. Almost all of the programs drew from 
other existing federal programs: team mem- 
bers in all but two courts travelled during the 
planning stage to observe at least one other 
district’s program in action. 

One-third of the program teams partic- 
ipated in a Center reentry team training 
program that emphasizes the collaborative 
and organizational elements of team supervi- 
sion. Other prominent sources of information 
and training were the National Association of 
Drug Court Professionals—with at least one 
team member in 62 percent of the programs 
attending one or more of its annual train- 
ing sessions; and the National Drug Court 
Institute, whose staff visited just under one- 
quarter of the program districts during their 
program development process. 


Type of Program and 

Eligibility Criteria 

Despite the cross-fertilization of pro- 
grams across districts, no two programs 
are identical. There are many sources of 
these differences, but most important from 
a design and research perspective is the 
program's goal as defined by the primary 
problem it is to address. Although the term 
“reentry” has been used widely in the judge- 
involved supervision context, the majority of 
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the 39 surveyed programs target substance 
abuse rather than prisoner reentry. 


Typology by Program Goals 


In the post-conviction context, reentry is 
defined as “the process of leaving prison and 
returning to society.” Reentry programs are 
therefore exclusively for ex-prisoners and are 
designed to address the whole range of issues 
critical to a prisoner's successful reintegration 
into the community. They are aimed at higher- 
risk offenders most in need of assistance, and 
target the services and oversight these higher- 
risk offenders receive based on a thorough, 


( Population Targeted by a 
Federal Judge-Involved 
Supervision Programs 

Other Specific 


Problem—General 
Population, 3 


individualized assessment of their crimino- 
genic risks and needs. 

There are five federal judge-involved reen- 
try programs that target higher-risk supervised 
releasees,° regardless of their substance abuse 
history or any other specific problem. Three 
of these five programs are limited to offend- 
ers newly received for supervision—a critical 


> J. Travis, But They All Come Back: Facing the 
Challenges of Prisoner Reentry, The Urban Institute 
Press, Washington, DC (2005), xxi. 


* In the federal system, “supervised release” is 


a separate sentence that may be imposed by the 
sentencing court to follow a term of imprison- 
ment. Supervised releasees, who are under the 
jurisdiction of the court, are therefore the focus 
of federal reentry programs. There are also a 
small number of parolees, sentenced before federal 
parole was abolished in 1987, who are received 
from prison for supervision each year. Parolees are 
supervised by federal probation officers, but fall 
under the jurisdiction of the United States Parole 
Commission—an executive branch agency—rather 
than the court. As such, they are generally excluded 
from judge-involved supervision programs. 


period in the prison-to-community transi- 
tion. The other programs are either open only 
to offenders with a particular type of problem 
and/or accept probationers (who are not 
returning from prison) as well as supervised 
releases. 

e Three reentry-drug focused programs 
target offenders who are returning from 
prison to the community, but only if they 
have a documented history of substance 
abuse. 

@ The majority of the programs—20—follow 
a general “drug court” model, available to 
probationers as well as supervised releasees 
as long as they have a documented history 
of substance abuse. (One of these operates 
as an alternative to revocation and targets 
only substance abusers who are charged 
with a violation.) 

e Three programs target probationers or 
supervised releasees who have problems 
other than substance abuse. One targets 
gangs, another mental health, and a third 
Native Americans who lack coping skills. 

e The remaining seven (7) programs are 
available to any higher-risk probationer 
or supervised release who meets the risk 
criteria set for the program. 

It is not surprising that so many of the 
programs focus on substance abuse given the 
reported success of drug courts at the state 
and local level, and the availability of train- 
ing from drug court professionals. There 
were, however, no significant differences 
across program types in the frequency with 
which they sought training from the National 
Association of Drug Court Professionals or 
the National Drug Court Institute during pro- 
gram development.’ 


Correlates of Program Types 

Looking at program eligibility criteria beyond 
the basic reentry and problem focus that 
define the different program types, there 
are only two that differentiated significantly 
among programs. As might be expected, pro- 
grams designed to address substance abuse or 
another specific type of problem are signifi- 
cantly less likely than others to target high-risk 
offenders per se, and the programs focused 
on substance abuse are more likely to accept 


Significance was tested with the chi-square sta- 
tistic. Given the small number of programs and the 
relatively large number of chi-square analyses per- 
formed, a result is reported as significant only if the 
significance level of the chi-square statistic reached 
the .01 level or beyond. Differences at or beyond the 
.05 level are reported as “tendencies.” 


offenders already under supervision in addi- 
tion to the new arrivals. 


Assessing Risk 


Across program types, 44 percent of the pro- 
grams target high-risk and 25 percent target 
moderate-risk offenders. The Risk Prediction 
Index (RPI)® score is used to determine the 
degree of risk in all but two of these pro- 
grams. RPI scores for program eligibility in 
the 27 programs that target high- or moder- 
ate-risk offenders range from 3-6 at the low 
end to 7-9 at the high end, with a span of 6-9 
the most common RPI risk criteria, adopted 
by seven programs. 

In addition to risk level criteria, some 
programs also have offense type eligibil- 
ity requirements to address the nature of 
a potential participant's risk. All types of 
offenders except sex offenders are eligible 
or eligible under certain circumstances to 
participate in the majority of programs, and 
28 percent of the programs do not automati- 
cally exclude sex offenders. The two program 
types that target substance abuse have a 
higher tendency to automatically exclude 
violent offenders, illegal aliens, and those 
with serious mental health problems.’ 

The actuarial risk levels adopted by the 
programs, and their inclusion of many very 
serious offenders, indicate that most aim their 
intensive programs at offenders at the higher 
levels of need and risk. On the other hand, 
most programs also predicate participation 
on some evidence that an offender is will- 
ing to change. All of the programs but two 
(one a reentry program, the other the Native 
American program) are voluntary; in 77 per- 
cent, the participant must have verbalized a 
readiness to change. 


Other Eligibility Limitations 


For practical reasons associated with the 
length and requirements of a program, the 
eligibility rules of a majority of the programs 
have a minimum term of supervision or 
time remaining under supervision (ranging 
from 12 to 36 months, with 24 months the 
most common) and include geographical 


* The RPI is a risk assessment device developed by 
the Federal Judicial Center and used since 1997 by 
the federal probation system to determine the gen- 
eral risk level of offenders received for probation or 
supervised release supervision. 


° The differences across all program types are not 
significant. When the program types are grouped 
as to whether or not they accepted only substance 
abusers, the differences were significant at the .05 
but not the .01 level. 
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by your program 


Q 9: What general level of offender risk is targeted 


Relationship Between Type Of Program And Other Eligibility Requirements’ 


Q 4: Does your program’s target population include:t 


@ High Risk @ Moderate Risk Any Risk @ Newly Received or Under Supervision Newly Received Only 
100% —— 100% 
80% 80% — 
60% — 60%|_ 
40% — 40% 
20% 20% 
0% 0% 
Higher Risk— Drug—Reentry Higher Risk— Substance Abuse —Otther Specific Higher Risk— Drug—Reentry Higher Risk— Substance Abuse —_ Other Specific 
Reentry Population Population(n=3) General Population Problem—General Problem—General Reentry Population Population(n=3) General Population Problem—General Problem—General 
(n=5) (n=7) Population (n=21) Population (n=3) (n=5 n=7 Population (n=21) Population (n=3 
=21.3 p<.01 = 39.0, p<.001 


*The graph displays the percentage of all programs of a particular type that target high, moderate or “any” risk and the percentage of programs that do or 
do not target offenders newly received for supervision. Given the small number of programs in each category, the percentages per se are clearly not stable, 
but are displayed for ease of comparison across type of program to indicate the source of the significant relationship. 

**Respondents were also given a choice of “Low Risk” but no one selected this answer. 


' The survey asked in a series of yes/no questions if the program targeted “Offenders Newly Received For Supervision,” “Offenders Under Supervision 
Charged with a Violation,” and/or “Offenders Under Supervision (regardless of violation status).” The graph combines the two “Under Supervision” 


(Participant Selection and 
Incentives to Participate 


Q17: Are final decisions to select program 
participants or, if voluntary, to offer the 
program to offenders made by: 


Team 
Majority, 5 


Program Judge, 8 


constraints that require participants to live 
in the jurisdiction where the court convenes 
and/or in an area where appropriate services 
are available. 


Selection Process 


The probation office is involved in the initial 
screening for participants in all programs, and 
solely responsible for making the preliminary 


recommendation in 28 (72 percent). Once 
screened as eligible, the final decision to 
accept or offer participation to an offender 
is most commonly made by team consensus, 
but five programs rely on team majority and 
in eight programs the program judge or the 
probation office makes the final selection. 


Incentives for Program Participation 


The 37 programs that are voluntary vary sig- 
nificantly in the percentage of offenders who 
agree to participate, from a reported low of 3 
percent to a reported high of 100 percent. The 
median across programs is 80 percent. The 
high volunteer rate in a number of programs 
is likely influenced by officer pre-screening of 
offenders on issues related to their likelihood 
of volunteering, e.g., assessing willingness to 
change before offering the program. 


Reduction in Supervision Term 


The primary incentive offered for participa- 
tion is a reduction in the term of supervision 
for offenders who graduate from the pro- 
gram.'’ Most commonly—in over 80 percent 
of the programs, the reduction is a year off, 


10 The one-year off is also offered by one of the two 
mandatory programs as an incentive for program 
completion. 


but there are some variations. In one pro- 
gram, the reduction varies depending on the 
length of the supervision term, e.g., 1 year off 
a 3-year term, 18 months off a 4-year term, 
2 years off a 5-year term. In others, supervi- 
sion is terminated regardless of the length of 
the remaining supervision term, either when 
the offender graduates from the program (1 
program) or after 12 months of successful 
post-graduation traditional supervision (4 
programs). In another program, each team 
member makes a recommendation of from 
one year off to immediate discharge after 12 
months of successful post-graduation tradi- 
tional supervision. 

There are also variations in the certainty 
of the reward. One aspect of certainty is 
whether offenders are told up front that, if 
they are successful, their supervision term 
will be reduced (55 percent of the programs) 
or that the reduction will be recommended to 
the court (45 percent of the programs). This 
feature of the reward is related to whether 
or not the program is headed by an active 
district judge, reflecting the reality that, in 
most circumstances, magistrate judges will 
be recommending the early termination of 
supervision to the district court judge with 
jurisdiction over the case. 


T 
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the 39 surveyed programs target substance 


abuse rather than prisoner reentry. 


Typology by Program Goals 

In the post-conviction context, reentry is 
defined as “the process of leaving prison and 
returning to society.” Reentry programs are 
therefore exclusively for ex-prisoners and are 
designed to address the whole range of issues 
critical to a prisoner's successful reintegration 
into the community. They are aimed at higher- 
risk offenders most in need of assistance, and 
target the services and oversight these higher- 
risk offenders receive based on a thorough, 


( Population Targeted by 
Federal Judge-Involved 


Supervision Programs 
Other Specific 
Problem—General 
Population, 3 


7A 


Higher Risk— 
General 
Population, 7 


individualized assessment of their crimino 
genic risks and needs. 

There are five federal judge-involved reen- 
try programs that target higher-risk supervised 
releasees,° regardless of their substance abuse 
history or any other specific problem. Three 
of these five programs are limited to offend- 


ers newly received for supervision—a critical 


They All Come Back: Facing the 
Challenges of Prisoner Reentry, The Urban Institute 
Press, Washington, DC (2005), xxi. 


J. Travis, But 


In the federal system, “supervised release” is 
a separate sentence that may be imposed by the 
sentencing court to follow a term of imprison- 
ment. Supervised releasees, who are under the 
jurisdiction of the court, are therefore the focus 
of federal reentry programs. There are also a 
small number of parolees, sentenced before federal 
parole was abolished in 1987, who are received 
from prison for supervision each year. Parolees are 
supervised by federal probation officers, but fall 
under the jurisdiction of the United States Parole 
Commission—an executive branch agency—rather 
than the court. As such, they are generally excluded 
from judge-involved supervision programs. 
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period in the prison-to-community transi- 
tion. The other programs are either open only 
to offenders with a particular type of problem 
and/or accept probationers (who are not 
returning from prison) as well as supervised 
releases. 

e Three reentry-drug focused programs 
target offenders who are returning from 
prison to the community, but only if they 
have a documented history of substance 
abuse. 

@ The majority of the programs—20—follow 
a general “drug court” model, available to 
probationers as well as supervised releasees 
as long as they have a documented history 
of substance abuse. (One of these operates 
as an alternative to revocation and targets 
only substance abusers who are charged 
with a violation.) 

e Three programs target probationers or 
supervised releasees who have problems 
other than substance abuse. One targets 
gangs, another mental health, and a third 
Native Americans who lack coping skills. 

e The remaining seven (7) programs are 
available to any higher-risk probationer 
or supervised release who meets the risk 
criteria set for the program. 

It is not surprising that so many of the 
programs focus on substance abuse given the 
reported success of drug courts at the state 
and local level, and the availability of train 
ing from drug court professionals. There 
were, however, no significant differences 
across program types in the frequency with 
which they sought training from the National 
Association of Drug Court Professionals or 
the National Drug Court Institute during pro- 
gram development. 


Correlates of Program Types 

Looking at program eligibility criteria beyond 
the basic reentry and problem focus that 
define the different program types, there 
are only two that differentiated significantly 
among programs. As might be expected, pro- 
grams designed to address substance abuse or 
another specific type of problem are signifi- 
cantly less likely than others to target high-risk 
offenders per se, and the programs focused 
on substance abuse are more likely to accept 


Significance was tested with the chi-square sta- 
tistic. Given the small number of programs and the 
relatively large number of chi-square analyses per 
formed, a result is reported as significant only if the 
significance level of the chi-square statistic reached 
the .01 level or beyond. Differences at or beyond the 


.05 level are reported as “tendencies.” 


offenders already under supervision in addi- 
tion to the new arrivals. 


Assessing Risk 

Across program types, 44 percent of the pro- 
grams target high-risk and 25 percent target 
moderate-risk offenders. The Risk Prediction 
Index (RPI)* score is used to determine the 
degree of risk in all but two of these pro- 
grams. RPI scores for program eligibility in 
the 27 programs that target high- or moder- 
ate-risk offenders range from 3-6 at the low 
end to 7-9 at the high end, with a span of 6-9 
the most common RPI risk criteria, adopted 
by seven programs. 

In addition to risk level criteria, some 
programs also have offense type eligibil- 
ity requirements to address the nature of 
a potential participant's risk. All types of 
offenders except sex offenders are eligible 
or eligible under certain circumstances to 
participate in the majority of programs, and 
28 percent of the programs do not automati- 
cally exclude sex offenders. The two program 
types that target substance abuse have a 
higher tendency to automatically exclude 
violent offenders, illegal aliens, and those 
with serious mental health problems.” 

The actuarial risk levels adopted by the 
programs, and their inclusion of many very 
serious offenders, indicate that most aim their 
intensive programs at offenders at the higher 
levels of need and risk. On the other hand, 
most programs also predicate participation 
on some evidence that an offender is will- 
ing to change. All of the programs but two 
(one a reentry program, the other the Native 
American program) are voluntary; in 77 per- 
cent, the participant must have verbalized a 
readiness to change. 


Other Eligibility Limitations 

For practical reasons associated with the 
length and requirements of a program, the 
eligibility rules of a majority of the programs 
have a minimum term of supervision or 
time remaining under supervision (ranging 
from 12 to 36 months, with 24 months the 


most common) and include geographical 


The RPI is a risk assessment device developed by 
the Federal Judicial Center and used since 1997 by 
the federal probation system to determine the gen 
eral risk level of offenders received for probation or 
supervised release supervision. 


The differences across all program types are not 
significant. When the program types are grouped 
as to whether or not they accepted only substance 
abusers, the differences were significant at the .05 
but not the .01 level. 
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QO 9: What general level of offender risk is targeted 


by your program?** 


@ High Risk ® Moderate Risk 


Q 4: Does your pro 


Any Risk 
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um’s target population include: 


eceived Only 


Higher Risk 


=21.3 p<.01 


**Respondents were also given a choice of “Low Risk” but no one selected this answer. 


39.0, p<.001 


*The graph displays the percentage of all programs of a particular type that target high, moderate or “any” risk and the percentage of programs that do o1 
do not target offenders newly received for supervision. Given the small number of programs in each category, the percentages per se are clearly not stable, 
but are displayed for ease of comparison across type of program to indicate the source of the significant relationship. 


"The survey asked in a series of yes/no questions if the program targeted “Offenders Newly Received For Supervision,’ “Offenders Under Supervision 
Charged with a Violation,’ and/or “Offenders Under Supervision (regardless of violation status).” The graph combines the two “Under Supervision 


categories. 


' Participant Selection and 
Incentives to Participate 
Q17: Are final decisions to select program 
participants or, if voluntary, to offer the 
program to offenders made by: 


Team 
Majority, 5 


Program Judge, 8 


) 


constraints that require participants to live 


in the jurisdiction where the court convenes 
and/or in an area where appropriate services 
are available. 


Selection Process 


The probation office is involved in the initial 
screening for participants in all programs, and 


solely responsible for making the preliminary 


recommendation in 28 (72 percent). Once 
screened as eligible, the final decision to 
accept or offer participation to an offender 
is most commonly made by team consensus, 
but five programs rely on team majority and 
in eight programs the program judge or the 
probation office makes the final selection. 


Incentives for Program Participation 

The 37 programs that are voluntary vary sig- 
nificantly in the percentage of offenders who 
agree to participate, from a reported low of 3 
percent to a reported high of 100 percent. The 
median across programs is 80 percent. The 
high volunteer rate in a number of programs 
is likely influenced by officer pre-screening of 
offenders on issues related to their likelihood 
of volunteering, e.g., assessing willingness to 


change before offering the program. 


Reduction in Supervision Term 

The primary incentive offered for participa- 
tion is a reduction in the term of supervision 
for offenders who graduate from the pro- 
gram.'” Most commonly—in over 80 percent 


of the programs, the reduction is a year off, 


The one-year off is also offered by one of the two 
mandatory programs as an incentive for program 
completion. 


but there are some variations. In one pro- 
gram, the reduction varies depending on the 
length of the supervision term, e.g., 1 year off 
a 3-year term, 18 months off a 4-year term, 
2 years off a 5-year term. In others, supervi- 
sion is terminated regardless of the length of 
the remaining supervision term, either when 
the offender graduates from the program (1 
program) or after 12 months of successful 
post-graduation traditional supervision (4 
programs). In another program, each team 
member makes a recommendation of from 
12 


months of successful post-graduation tradi- 


one year off to immediate discharge after 


tional supervision. 

There are also variations in the certainty 
of the reward. One aspect of certainty is 
whether offenders are told up front that, if 
they are successful, their supervision term 
will be reduced (55 percent of the programs) 
or that the reduction will be recommended to 
the court (45 percent of the programs). This 
feature of the reward is related to whether 
or not the program is headed by an active 
district judge, reflecting the reality that, in 
most circumstances, magistrate judges will 
be recommending the early termination of 
supervision to the district court judge with 


jurisdiction over the case. 
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@ Will Reduce 


Q 54: Are offenders told when deciding about participating in the program that, aes, 
program graduation, his/her sentence will be reduced or that there will be a 
recommendation that the sentence be reduced? 


Recommend Reduction 


100% -— 


80% 


60% 


40% 


20 


0% 


Active District Court Judge 
(n=20) 


x2=6.7, p=.01 


No Active District Court Judge 
(n=18) 


A second element of certainty is the vesting 
of the reduction. In 55 percent of the pro- 
grams, the reduction vests upon a successful 
offender's graduation, but 
© in 26 percent there are other “good behav- 

ior” requirements (e.g., additional months 

of sobriety or successful supervision) that 
must be satisfied after graduation, and 

© in 18 percent the reduction is granted at 
graduation but does not vest, i.e., it can be 
rescinded for subsequent noncompliance. 

Vesting at the time of graduation is no 
more likely in programs in which offenders 
are told up front that they will get the reduc- 
tion than in those in which the promise is for a 
recommendation; and neither of the certainty 
elements correlates with reported volunteer 
rates.'! 


Less Adversarial Procedures 


All but one of the programs are struc- 
tured to use graduated sanctions under less 
adversarial procedures to address offender 
noncompliance of varying degrees.'* These 
programs require participants to consent to 
the imposition of specified sanctions by the 
program judge under the less formal pro- 
gram process. This allows a broader range of 


'' “Reported” is emphasized given that some pro- 


grams reported 100% volunteer rates, which may 
reflect a practice of only offering the opportunity 
to participate to those deemed likely to volunteer in 
the first place. See discussion on page 6, infra. 


'> Most of the interagency agreements contain 


language to the effect that, with some exceptions, 
conduct that would otherwise constitute a supervi- 
sion violation and lead to a violation hearing may be 
handled in an informal manner. 


sanctions—up to and including short jail time’? 
—to be implemented without delay while 
maintaining the offender in the program 
and outside of normal violation procedures. 
‘‘Noncompliance involving violations of at 
least moderate severity are handled under 
less formal program procedures in 79 percent 
of the programs, including 18 percent that 
handle high severity violations within the 
program. 

Just over one-third of the programs also 
utilize less formal hearings conducted within 
the program to address whether a charged vio- 
lation occurred if disputed by the offender. Of 
the 25 programs that instead refer contested 
noncompliance issues for consideration under 
regular violation procedures, three-quarters 
either maintain the offender in the program or 
permit return regardless of the outcome. 

Most programs maintain commitment to 
the eventual reintegrative goal even when 
revocation is necessary, with 79 percent 
permitting—and one requiring—a revoked 


'* In most programs (82 percent), the offender 


agrees at program start that jail time is one of the 
sanctions that may be imposed. The range in the 
amount of jail time was from 1 to 30 days across 
the programs, with the majority—20 of the 32 
programs that have a special jail time provision—at 
7 days. 


'* Supervision policies require probation officers 


to utilize graduated sanctions in response to non- 
compliance. If, however, the desired sanction affects 
the length or conditions of the term of supervision, 
officers must petition the sentencing judge for a 
modification—an action that may be contested by 
the offender under the procedures of Fed. R. Crim. 
P. 32.1(b). This can obviously affect the timeliness 
and certainty of such sanctions. 


offender to return to the program upon 
re-release. 


Program Structure 


Length 


Absent noncompliance, the amount of time 
from program start to graduation ranges 
across programs from 5 to 18 months, with 12 
months—the “best case” duration of 33 pro- 
grams—by far the most common. In keeping 
with the commitment to success, all programs 
permit offenders who have some difficulty 
meeting supervision goals but remain engaged 
to take longer—usually up to 18 months—to 
meet graduation requirements. 


Phases 


The majority of programs have just one pre- 
graduation phase, during which offenders 
meet regularly with the team weekly (1 pro- 
gram), bi-weekly (4 programs), monthly (15 
programs), or quarterly (1 program); but 10 
of these programs have a post-graduation 
phase requiring that offenders complete an 
additional 12 months of successful tradi- 
tional supervision after graduation to earn the 
reduction in their supervision term. 

There are 17 programs that have from 
two to four pre-graduation phases that step 
offenders through decreasing levels of inter- 
vention and oversight (e.g., program team 
sessions, reporting to the probation offi- 
cer, drug testing) as the participants reach 
increasingly demanding benchmarks of suc- 
cess (e.g., finding a job, attending program 
and treatment sessions as required, remaining 
drug-free for specified periods of time). Of the 
17 programs with pre-graduation phases, all 
but three require offenders to meet with the 
program team weekly or bi-weekly during the 
first phase, and reduce to monthly sessions by 
the last phase. 


Interim Rewards 


All but two programs offer interim rewards 
before graduation, most commonly public 
acknowledgement of success and written 
acknowledgement of achievement. Over 
half—56 percent—of the programs also give 
token gifts, and one-third offer phase advance- 
ment or other reduced reporting requirements 
as offenders meet interim goals throughout 
the program. These are used as tools to shape 
pro-social behavior and to encourage active 
participation and program completion. 
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Team Composition 


Collaborative teamwork is a key element of 
the drug court movement that gave impetus 
to judge-involved supervision programs. All 
of the surveyed programs operate with a 
team, comprising—at a minimum—a pro- 
gram judge and federal probation office staff. 
Three programs have no other team members, 
whereas ten of the surveyed programs feature 
a full team comprising a prosecutor, defense 
counsel, and service provider in addition 
to the judge and probation office staff who 
participate in all program tasks. These tasks 
include selecting program participants, staff- 
ing cases with the team before court sessions 
with offenders, attending court sessions with 
offenders, and helping to select appropriate 
rewards for offender achievements and sanc- 
tions for offender noncompliance. 


Service providers are far less likely than 
either of the attorneys to play any of the key 
program roles—particularly selecting partici- 
pants. When the programs were grouped by 
whether or not they targeted substance abuse, 
those that did were significantly more likely to 
include a service provider as a team member 
and to have that service provider perform all 
program tasks. This highlights that the role, if 
any, of a service provider on the team—and 
the type of provider included—reflects as it 
should the nature of the population being 
served. 


Program Responsibilities and 
Traditional Roles 


Judge-involved supervision programs are 
designed to provide for collaborative, inten- 
sive team supervision and therefore add 


@ Attorneys 
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Attend Court 
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2=8.2, p<.01 2=8.1, p<.01 


responsibilities to the traditional roles played 
by the judges, probation office staff, attorneys, 
and service providers involved. Some of these 
responsibilities are natural extensions of tra- 
ditional roles, while others require new skills 
and ways of thinking. All require additional 
time and attention. 


Judges 


Most judge-involved supervision programs 
are specifically designed to move judges away 
from their independent adjudicative role to 
one in which they collaborate with criminal 
justice and community service professionals 
in a less formal team approach to monitoring 
and motivating offenders’ pro-social behavior 
and personal growth. Program judges under- 
take these responsibilities in addition to their 
regular duties, with none receiving a reduction 
in case assignments. 

In the 39 programs surveyed, 64 judges 
were involved routinely as the program 
judge—25 active district judges, 6 senior 
district judges and 33 magistrate judges.'* An 
additional 23 judges participated occasionally 
as backup program judges. 

Depending on the program, program 
judges schedule sessions with offenders 
weekly (18 percent), bi-weekly (36 percent) 
or monthly (46 percent). The sessions are 
scheduled to last anywhere from 30 minutes 
to 3 hours, with most programs setting aside 
an hour. From 3 to 30 offenders participate in 
each session, with a median of 10 participants 
across programs. 


Collaborative Decision-Making 


Most program judges are part of a collabora- 
tive decision-making structure. In all but two 
of the programs, the judge convenes a case 
conference among team members prior to 
each court session to discuss offenders’ prog- 
ress and appropriate rewards and sanctions. 
Depending on the program, these sessions are 
scheduled from 15 minutes to 2 hours, with 
most programs setting aside an hour. 

A consensus process is used in the major- 
ity of programs to select appropriate rewards, 
but the sanctioning decision in the majority 
of programs is left to the judge with input 
from the team. This style of decision-making 
provides for collaborative input, but recog- 
nizes that both program requirements and 
broader federal supervision policy require an 
appropriate, proportional response to each 
act of offender noncompliance. Where there 


There are more judges than programs because 
13 of the programs have more than one team. 
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Decision-Making Style 


Q 58: Are decisions about 
appropriate interim rewards 
made by: 


Q 61: Are decisions about 
appropriate sanctions within 
the program made by: 


Program Judge, 23 


is disagreement among team members as to 
what is appropriate, someone has to decide if 
there was noncompliance and, if so, the appro- 
priate sanction. These decisions fall within the 
traditional role of the judge; no other team 
member has the authority to impose the full 
range of program sanctions. 


Informality 


Despite the less formal structure of the 
programs, almost three-quarters of the 64 
program judges conduct the offender sessions 
robed, and 30 of these are also seated on the 
bench in the traditional court setting. Other 
judges mix elements of formality, wearing 
robes but sitting at a conference table with 
offenders and other team members, while 
only five (5) sit at a conference table with the 
others in street clothes rather than judge's 
robes—the least formal configuration. 


Probation Office Staff Officers 


The role of probation officers in judge-involved 
supervision programs fits squarely within 
their traditional and statutory responsibilities 
(18 U.S.C. § 3603). These duties include keep- 
ing the court informed of the conduct and 
condition of a person under supervision and 


‘© Survey question 32 asked if, during the court ses- 
sion, the judge was on the bench and the offenders 
in the jury box or the judge is seated at a conference 
table with the others or “Other Configuration.” The 
“Other” text field sometimes included information, 
sometimes not. Further, the survey did not ask 
for the style of each program judge, so the style of 
judges in multi-judge programs could be extrapo- 
lated only if specific information was provided in 
comments to this section. 


using “all suitable methods, not inconsistent 
with the conditions specified by the court” 
to aid a person under supervision and “bring 
about improvements in his conduct and con- 
dition.” Further, current supervision policies 
are based on the same principles that under- 
gird judge-involved supervision programs: 
targeting supervision resources at higher-risk 
offenders, targeting the services these higher- 
risk offenders receive to their criminogenic 
risks/needs, and transitioning to less structure 
throughout the period of supervision. 

The judge-involved supervision programs 
add to what would be expected from a regu- 
lar supervision plan the requirement that 
the offender report to the program team at 
regular, specified intervals. In addition, unlike 
the general guidance provided by supervision 
policies, many programs set specific contact 
levels for offender reporting to the probation 
office, many of which start quite high (weekly 
or bi-weekly) and decrease over time. These 
specific requirements may or may not be dif- 
ferent from what would have been required 
absent the program, but are likely to be more 
intensive for officer and offender alike. The 
collaborative team approach also adds to the 
officer's role much of the responsibility for 
coordinating, if not the team itself, then the 
information that is reported to the team on a 
regular basis. 

In the 39 programs surveyed, 101 probation 
officers—64 line officers and 37 specialists— 
supervise program offenders, with a range of 
from 1 to 8 officers across programs. Two- 
thirds of the programs have either one officer 
(11 programs) or two officers (15 programs) 


involved. Most (90 percent) of these officers 
are not dedicated to the program but also 
supervise other offenders. 

Perhaps because of the prevalence of mixed 
caseloads, three-quarters of the programs do 
not limit the number of program offenders or 
total (program and non-program) offenders 
that may be supervised by an officer involved 
in the program at any given time. In the few 
programs that set caseload limits, the most 
common was a limit of 20 program offenders 
(with a range of from 5 to 25 across 8 pro- 
grams) and a total caseload limited to 35-40 
offenders. 

Questions about the size and structure of 
officers’ caseloads must be assessed carefully 
to be sure that officers have adequate time to 
carry out program tasks, fieldwork demands, 
and the supervision of non-program offend- 
ers who do not have such a high profile with 
the court. This is particularly important if the 
program excludes the riskiest offenders, who 
will require an equally intense, if perhaps dif- 
ferently focused, plan of action. 


Managers, Supervisors, and Support Staff 


In addition to the officers who provide super- 
vision, other probation personnel often serve 
as team members or provide support to the 
team. Office managers (the chief, deputy chief, 
and/or assistant deputy chief) routinely attend 
court sessions with offenders and/or the plan- 
ning conference in 17 of the programs, and 
supervisors attend in 23. Other staff, including 
backup officers, specialists, and administra- 
tive personnel, routinely participate in three 
programs each. This is in keeping with the 
traditional role of managers, supervisors, spe- 
cialists, and others to support the work of 
officers, but in a different and more public 
context. 


Counsel 


A primary goal of federal supervision in gen- 
eral and judge-involved supervision programs 
in particular is to facilitate offenders’ success 
in maintaining a law-abiding lifestyle as con- 
tributing members of the community both 
during the period of supervision and beyond. 
This does not contradict prosecutorial pub- 
lic safety goals or defense goals to promote 
clients’ interests, but is a shift from the cus- 
tomary prosecutorial focus on pursuing public 
safety by conviction and punishment, and the 
defense focus on serving their clients through 
advocacy and protection of rights. Further, 
the direct non-adversarial contact among 
team members and with offenders is a new 
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46: Is a defender team member 
expected to represent program offenders 
alleged to have violated program rules 

or conditions of supervision? 


Yes, In Program 
Only, 10 


role for counsel, as is the expectation that 
they are to help shape a program participant's 
behavior by administering rewards and gradu- 
ated sanctions—rather than traditional legal 
consequences alone—to promote new ways 
of thinking and positive behavioral outcomes. 


U.S. Attorneys 


Representatives from the U.S. Attorney’s 
Office played some role in 35 of the 39 pro- 
grams surveyed and were full team partners 
playing each key role in 24. Few programs 
expect U.S. attorneys also to perform tradi- 
tional investigative or evidentiary presentation 
tasks in their role as a team member. Only 


three programs expect them to investigate. 


allegations of noncompliance, and five expect 
them to assist in the presentation of evidence 
at informal program court sessions to address 
noncompliance. 


Defense Counsel 


Defense counsel—usually representatives 

from the Federal Public Defender’s Office— 

played some role in 36 of the 39 programs and 

were full team partners in 23. The programs 

are split as to the extent to which defense 

counsel are also called upon to play their tra- 

ditional roles. 

© In 56 percent of the programs, the defender 
representative advises offenders who have 
been offered a slot in the program as to 
the pros and cons of participation; in 44 
percent they do not. 

e The same 56 percent majority (although 
not the same programs) expect the 


defender to represent program offenders 
charged with noncompliance during either 
the informal program sanctioning process 
or all (formal and informal) phases of the 
violation process, while in 44 percent of 
the programs, there is no attorney-client 
relationship between program offenders 
and the defense representative on the team. 
One program started out with the defender 
as offender representative, but moved to 
having no attorney-client relationship. 
These varied approaches to the role 
of defense counsel highlight the need for 
each program to address how to handle the 
potential conflict that can arise when the 
same attorney is both a team member—with 
responsibilities to the team—and also some- 
one with responsibilities to clients defined by 
professional ethics. 


Service Providers 


The traditional role of community service 
providers is to provide service, prepare reports 
for the probation officer on offenders’ status 
and response to treatment, and recommend 
appropriate modifications to treatment plans. 
These tasks continue in any judge-involved 
supervision program, but the reports and 
recommendations would be more frequent to 
accommodate the program's session schedule. 

Providers who are team members take 
on additional tasks of participating in face- 
to-face discussions with the team and the 
offender to, as needed, review the status of a 
particular offender or offer a broader perspec- 
tive outside the criminal justice system and/ 
or perhaps even facilitate the team process 
itself. It is particularly important to clarify the 
role(s) a service provider is expected to play in 
programs that do not target one specific type 
of problem. 


Resource Commitment 


Time commitment to the program will 
depend, in part, on the number and composi- 
tion of the supervision teams, the frequency 
and length of the court sessions with offenders 
and who attends, and whether the team holds 
pre-session preparatory conferences before 
the court sessions and, if so, how much time 
is set aside.'” 

We combined program information with 
team composition information to estimate the 


‘7 These activities do not take into consider- 
ation the time it takes to coordinate the sessions 
logistically, assemble the relevant paperwork, or 
undertake additional supervision activities that 
might be entailed. 


amount of time that a single team'* devotes 
to conducting and meeting to prepare for 
sessions with offenders during the year. The 
number of routine attendees at the court 
session was multiplied by the time usually 
scheduled for these sessions,'? and the result 
then multiplied by the number of sessions 
scheduled during the year, ie., 52 if held 
weekly, 26 if bi-weekly, and 12 if monthly. 
The attendee count includes one judge and 
one probation officer plus any prosecutors, 
defense counsel, service providers,” and other 
probation office staff reported as routinely 
attending these sessions. We repeated the 
calculations for the preparatory meetings and 
added the two for an estimated total time 
commitment. 


Time by Task and Participant 


The range in the estimated time required by 
each team per year for basic program tasks 
varies across program from a low of 39 hours 
to a high of 936 hours, with the totals resulting 
from various combinations of program design 
decisions. 

The low-end annual team time commit- 
ment, estimated at 39 hours, is for a program 
in which the judge, probation officer, and 
supervisory probation officer meet every 
other week for 30 minutes with an average 
of 12 offenders. There is no preparatory team 
conference. The estimated per team judge 
time commitment is 1.1 hours per month. 

Two programs share the high-end 936-hour 
annual time commitment. One is a program 
in which an expanded team (full team plus a 
probation supervisor) meets weekly for two 
hours with, on average, 12 offenders. Each 
session is preceded by a one-hour preparatory 
conference. The per team judge time com- 
mitment is 13 hours per month. The second 
is a program that features weekly one-hour 
sessions with, on average, five (5) offenders 
and is preceded by a one-hour preparatory 
conference. The high time commitment stems 
primarily from the involvement of up to four 
(4) additional probation office staff in both 
the court session and preparatory conference. 


Average Time Across Programs 


The average estimated total time across all 
programs is 321 hours annually (median = 


‘8 Thirteen (13) programs have more than one 
team. 


'% Survey Question 30. If a range of time was 


reported, the lower end was used for the time 
estimate. 


0 Survey Question 41. 
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Decision-Making Style 


Q 58: Are decisions about 


made by: 


Program Judge, 11 


is disagreement among team members as to 
what is appropriate, someone has to decide if 
there was noncompliance and, if so, the appro 
priate sanction. These decisions fall within the 


traditional role of the judge; no other team 


member has the authority to impose the full 


range of program sanctions. 


Intormalit, 


Despite the less formal structure of the 
programs, almost three-quarters of the 64 
program judges conduct the offender sessions 
robed, and 30 of these are also seated on the 
bench in the traditional court setting. Other 
judges mix elements of formality, wearing 
robes but sitting at a conference table with 
offenders and other team members, while 
only five (5) sit at a conference table with the 
others in street clothes rather than judge's 
robes—the least formal configuration.” 


Probation Oftice Statf Officers 


[he role of probation officers in judge-involved 
supervision programs fits squarely within 
their traditional and statutory responsibilities 
18 U.S.C. § 3603). These duties include keep 
ing the court informed of the conduct and 


t 
condition of a person under supervision and 


Survey question 32 asked if, during the court ses 
sion, the judge was on the bench and the offenders 
in the jury box or the judge is seated at a conference 
table with the others or “Other Configuration.” The 

Other 


sometimes not 


text field sometimes included information 


Further, the survey did not ask 
for the stvle of each program judge, so the stvle of 
judges in multi-judge programs could be extrapo 


lated only if specific information was provided in 


comments to this section 


QO 61: Are decisions about 
appropriate sanctions within 
the program made by 


Program Judge, 23 


using “all suitable methods, not inconsistent 
with the conditions specified by the court’ 
to aid a person under supervision and “bring 
about improvements in his conduct and con 
dition.” Further, current supervision policies 
are based on the same principles that under- 
gird judge-involved supervision programs: 
targeting supervision resources at higher-risk 
offenders, targeting the services these higher- 
risk offenders receive to their criminogenic 
risks/needs, and transitioning to less structure 
throughout the period of supervision 

The judge-involved supervision programs 
add to what would be expected from a regu 
lar supervision plan the requirement that 
the offender report to the program team at 
regular, specified intervals. In addition, unlike 
the general guidance provided by supervision 
policies, many programs set specific contact 
levels for offender reporting to the probation 
office, many of which start quite high (weekly 
or bi-weekly) and decrease over time. These 
specific requirements may or may not be dif- 
ferent from what would have been required 
absent the program, but are likely to be more 
intensive for officer and offender alike. The 
collaborative team approach also adds to the 
officer's role much of the responsibility for 
coordinating, if not the team itself, then the 
information that is reported to the team on a 
regular basis. 

In the 39 programs surveyed, 101 probation 


otficers—64 line officers and 37 specialists— 


supervise program offenders, with a range of 


from | to 8 officers across programs. ‘Two 
thirds of the programs have either one officer 


(11 programs) or two officers (15 programs) 
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involved. Most (90 percent) of these officers 
are not dedicated to the program but also 
supervise other offenders 

Perhaps because of the prevalence of mixed 
caseloads, three-quarters of the programs do 
not limit the number of program offenders o1 
total (program and non-program) offenders 
that may be supervised by an officer involved 
in the program at any given time. In the few 
programs that set caseload limits, the most 
common was a limit of 20 program offenders 
(with a range of from 5 to 25 across 8 pro 
grams) and a total caseload limited to 35-40 
offenders. 

Questions about the size and structure of 
officers’ caseloads must be assessed carefully 
to be sure that officers have adequate time to 
carry out program tasks, fieldwork demands 
and the supervision of non-program offend 
ers who do not have such a high profile with 
the court. This is particularly important if the 
program excludes the riskiest offenders, who 
will require an equally intense, if perhaps dif 


ferently focused, plan of action. 


AA 


Vanagers, Supervisors, and Support Statt 


In addition to the officers who provide super 
vision, other probation personnel often serve 
as team members or provide support to the 
team. Office managers (the chief, deputy chiet 
and/or assistant deputy chief) routinely attend 
court sessions with offenders and/or the plan 
ning conference in 17 of the programs, and 
supervisors attend in 23. Other staff, including 
backup officers, specialists, and administra 
tive personnel, routinely participate in three 
programs each. This is in keeping with the 
traditional role of managers, supervisors, spe 
cialists, and others to support the work of 
officers, but in a different and more public 


context 


Counsel 


A primary goal of federal supervision in gen 
eral and judge-involved supervision programs 
in particular is to facilitate offenders’ success 
in maintaining a law-abiding lifestyle as con 
tributing members of the community both 
during the period of supervision and beyond. 
This does not contradict prosecutorial pub- 
lic safety goals or defense goals to promote 
clients’ interests, but is a shift from the cus 
tomary prosecutorial focus on pursuing public 
safety by conviction and punishment, and the 
detense focus on serving their clients through 
advocacy and protection of rights. Further, 
the direct non-adversarial 


contact among 


team members and with offenders is a new 
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O 46: 


expected to represent program offenders 


Is a defender team member 


alleged to have violated program rules 
or conditions of supervision? 


Yes, In Program 
Only, 10 


role for counsel, as is the expectation that 
they are to help shape a program participant's 
behavior by administering rewards and gradu 


ated sanctions—rather than traditional legal 


5 


consequences alone—to promote new ways 


of thinking and positive behavioral outcomes. 


U.S. Attorneys 


Representatives from the U.S. Attorney's 
Office played some role in 35 of the 39 pro- 
grams surveyed and were full team partners 
playing each key role in 24. Few programs 
expect U.S. attorneys also to perform tradi- 
tional investigative or evidentiary presentation 
tasks in their role as a team member. Only 
three programs expect them to investigate 
allegations of noncompliance, and five expect 
them to assist in the presentation of evidence 
at informal program court sessions to address 


noncompliance. 


Defense Counsel 


Defense counsel—usually representatives 
from the Federal Public Defender’s Office- 
played some role in 36 of the 39 programs and 
were full team partners in 23. The programs 
are split as to the extent to which defense 
counsel are also called upon to play their tra- 
ditional roles. 

@ In 56 percent of the programs, the defender 
representative advises offenders who have 
been offered a slot in the program as to 
the pros and cons of participation; in 44 
percent they do not. 

e The same 56 percent majority (although 
not the same 


programs) expect the 


JUDGE-INVOLVED SUPERVISION PROGRAMS 43 


defender to represent program offenders 
charged with noncompliance during either 
the informal program sanctioning process 
or all (formal and informal) phases of the 
violation process, while in 44 percent of 
the programs, there is no attorney-client 
relationship between program offenders 
and the defense representative on the team. 
One program started out with the defender 
as offender representative, but moved to 
having no attorney-client relationship. 
These varied approaches to the role 
of defense counsel highlight the need for 
each program to address how to handle the 
potential conflict that can arise when the 
same attorney is both a team member—with 
responsibilities to the team—and also some 
one with responsibilities to clients defined by 


professional ethics. 


Service Providers 
The traditional role of community service 
providers is to provide service, prepare reports 
for the probation officer on offenders’ status 
and response to treatment, and recommend 
appropriate modifications to treatment plans. 
These tasks continue in any judge-involved 
supervision program, but the reports and 
recommendations would be more frequent to 
accommodate the program's session schedule. 
Providers who are team members take 
on additional tasks of participating in face- 
to-face discussions with the team and the 
offender to, as needed, review the status of a 
particular offender or offer a broader perspec- 
tive outside the criminal justice system and/ 
or perhaps even facilitate the team process 
itself. It is particularly important to clarify the 
role(s) a service provider is expected to play in 
programs that do not target one specific type 


of problem. 


Resource Commitment 


Time commitment to the program will 
depend, in part, on the number and composi- 
tion of the supervision teams, the frequency 
and length of the court sessions with offenders 
and who attends, and whether the team holds 
pre-session preparatory conferences before 
the court sessions and, if so, how much time 
is set aside. 

We combined program information with 


team composition information to estimate the 


These activities do not take into consider 
ation the time it takes to coordinate the sessions 
logistically, assemble the relevant paperwork, or 
undertake additional supervision activities that 


might be entailed. 


amount of time that a single team'* devotes 
to conducting and meeting to prepare for 
sessions with offenders during the year. The 
number of routine attendees at the court 
session was multiplied by the time usually 
scheduled for these sessions,'’ and the result 
then multiplied by the number of sessions 
scheduled during the year, ie., 52 if held 
weekly, 26 if bi-weekly, and 12 if monthly. 
The attendee count includes one judge and 
one probation officer plus any prosecutors, 
defense counsel, service providers,” and other 
probation office staff reported as routinely 
attending these sessions. We repeated the 
calculations for the preparatory meetings and 
added the two for an estimated total time 
commitment. 


Time by Task and Participant 

The range in the estimated time required by 
each team per year for basic program tasks 
varies across program from a low of 39 hours 
to a high of 936 hours, with the totals resulting 
from various combinations of program design 
decisions. 

The low-end annual team time commit- 
ment, estimated at 39 hours, is for a program 
in which the judge, probation officer, and 
supervisory probation officer meet every 
other week for 30 minutes with an average 
of 12 offenders. There is no preparatory team 
conference. The estimated per team judge 
time commitment is 1.1 hours per month. 

Two programs share the high-end 936-hour 
annual time commitment. One is a program 
in which an expanded team (full team plus a 
probation supervisor) meets weekly for two 
hours with, on average, 12 offenders. Each 
session is preceded by a one-hour preparatory 
conference. The per team judge time com- 
mitment is 13 hours per month. The second 
is a program that features weekly one-hour 
sessions with, on average, five (5) offenders 
and is preceded by a one-hour preparatory 
conference. The high time commitment stems 
primarily from the involvement of up to four 
(4) additional probation office staff in both 
the court session and preparatory conference. 


Average Time Across Programs 
The average estimated total time across all 
programs is 321 hours annually (median = 


hirteen (13) programs have more than one 
team. 

Survey Question 30. If a range of time was 
reported, the lower end was used for the time 


estimate. 


Survey Question 41. 
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at Least One Graduation 
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( Graduation Rates for Those 32 Programs in Operation Long Enough to Have Had 7 
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Team member # of Programs in Average Number Median Number of 

which team member of Hours Spent Hours Spent Annually 
played any role Annually on Program | on Program Sessions 
‘Sessions / Meetings / Meetings 

Judges 39 53 39 

Probation Staff 39 127 104 

U.S. Attorney’s 50 

Office 

Defense Counsel 36 51 43.5 

Service Provider 26 40 50 


260 hours annually). Judges average approxi- 
mately 53 hours a year, or 4.4 hours per month 
(median = 39 hours annually, or 3.2 hours per 
month). The probation time is the greatest 
because multiple staff often attend the court 
sessions (31 programs) and/or the preparatory 
conferences (28 programs). 

Neither average estimated total time nor 
judge time were related to program type, but 


programs in which active district judges serve 
as program judges committed significantly 


less time than others. 


Program Reach and 

Graduation Rates 

At the time of the survey, a total of 1,413 
offenders had participated in the 39 programs. 
The range across programs was from 4 to 160, 


depending on the length of time the program 
had been in operation and the number of 
participants at any one time. The range in the 
number of offenders who participate (or, for 
newer programs, are expected to participate) 
at any given time is from 5 to 55 (median of 10 
participants), affected primarily by the num- 
ber of judges—and therefore teams—involved 
in the program. 

Of the 1,413 participants, 839 were no lon- 
ger in the program, having either graduated, 
quit the program, terminated unsuccessfully, 
or been administratively discharged (e.g., 
became ill, transferred out of district). Of 
these former participants, 422 graduated—but 
not all programs had been operating long 
enough at the time of the survey to have had 
a graduation. Based on the 32 programs that 
were operating long enough to have had at 
least one graduation (821 former partici- 
pants in all), the overall graduation rate was 
51.4 percent. Across these programs, the rate 
ranged from 0 to 88 percent, with a median of 
40.5 percent. 
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Implications for Further 
Research 


This examination of judge-involved super- 
vision programs indicates a wide range of 
goals, philosophies, and design features that 
affect the public resources a program requires 
and the level of intervention with offender 
participants. The next step is to get a bet- 
ter understanding of how various program 
features are related to program success. Do 
participants in some programs adjust better 


than those in other programs? If so, is this 
due to characteristics of the offenders who 
are in the program or to characteristics of the 
program itself—or to some combination of 
the two? 

To begin to answer these questions with 
respect to existing programs, we would need 
to examine the relationship between supervi- 
sion outcomes (e.g., re-arrests, termination 
status) and information from the survey about 
key program design features (e.g., program 


type, team composition, time commitment 
and decision-making style), while controlling 
for offender characteristics known to affect 
supervision outcomes (e.g., RPI score, nature 
of substance abuse, employment status). 

The Center is currently working with the 
Office of Probation and Pretrial Services to 
assess the feasibility of combining survey data 
with information from the Probation and 
Pretrial Services Automated Case Tracking 
System (PACTS) to explore these relationships. 


Average Estimated Annual Hours Committed to Court Sessions with Offenders and Preparatory 
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OVER THE PAST SEVERAL years the 
idea of evidence-based practice (EBP) has 
exploded within community corrections, and 
in the federal probation and pretrial services 
system in particular. For most, the idea of 
evidence-based practice focuses on using 
evidence to determine effective programs 
and practices for defendants and offenders. 
However, evidence-based practice is really a 
more expansive concept: it is the idea of using 
evidence (research) in all operations. This 
article focuses on using the concept of EBP in 
the implementation of any program, practice, 
or change within a system. In other words, 
what does the research tell us about how to 
effectively implement change? 


The Science of Implementation 


Many may be surprised to learn that there is 


a significant literature documenting imple- - 


mentation strategies and their effectiveness. In 
fact, there is a comprehensive synthesis of the 
literature on implementation (Fixsen, Naoom, 
Blase, Friedman, and Wallace, 2005) that pro- 
vides guidance to those hoping to effectively 
implement change in their organization. First, 
it must be recognized that implementation 
is a process that will take several years to 
accomplish. How do you know when you 
have achieved implementation? The National 
Implementation Research Network (NIRN), 
part of the University of North Carolina at 
Chapel Hill (http://www.fpg.unc.edu/~nirn/ 
default.cfm), suggests that new ways of work- 
ing are fully implemented when 50 percent of 
staff meet performance criteria for a specific 
skill, and the program or practice has reached 
scale when 60 percent of the population who 
could benefit are actually receiving the service 
(Van Dyke, 2011, personal communication). 
Of the many interventions that have come 


into the federal system over the past several 
years (risk assessment, cognitive-behavioral 
interventions, workforce development pro- 
grams, reentry courts, etc.), only a scant few 
would meet the NIRN criteria for being fully 
implemented and “scaled up.” 


Why is It so Hard to Implement? 


Effective implementation requires at least 
three components: fidelity, sufficient scale, 
and sustainability. First, the program/practice 
must be implemented with fidelity. In other 
words, are you doing it right? The answer to 
this question may seem relatively straightfor- 
ward for some issues (such as scoring a risk 
assessment correctly) but may be quite com- 
plex for others, such as cognitive-behavioral 
interventions (CBI). To truly measure the 
fidelity of something like CBI, you have to 
listen to the conversation, either in person 
or via audio/videotape, and code whether 
the officer/contract vendor is following the 
components of CBI. Few districts currently 
have the capacity for that level of measure- 
ment. Even if a district can determine that 
a program or service is being administered 
with fidelity, it is often only a few officers who 
do it, or a few contract vendors that provide 
the service, thus making it difficult to provide 
the program or service to at least 60 percent 
who would benefit from the service. Finally, 
it is difficult to sustain programs or services 
due to staff turnover, changes in contracts, 
decreased interest, or other factors. All of 
these issues merge to make effective imple- 
mentation a difficult endeavor. 


What Doesn’t Work for 
Implementation? 


As with the literature on offender program- 
ming, so in the area of implementation there 


Melissa Alexander 
Chief U.S. Probation Officer, Middle District of North Carolina 


is more information on what doesn’t work 
than on what does. Typically, changes in 
the correctional system seem to be man- 
dated (by the Administrative Office, judges, 
chief probation officers), and research dem- 
onstrates that mandating change in and of 
itself is ineffective. What else is ineffective? 
Two of the most common practices in our 
system: providing information and training! 
Research consistently shows, throughout a 
multitude of human service industries, that 
these two components without the additional 
supports outlined below will not lead to any 
significant, sustained changes in programs 
or services (Fixsen, Naoom, Blase, Friedman, 
and Wallace, 2005). 


Effective Implementation: 
Using Drivers and Stages 


In order to increase the likelihood of effec- 
tively implementing a new program or 
practice, the implementation must be com- 
prehensive and well thought out. Fortunately, 
NIRN has also provided a significant amount 
of resources to assist this endeavor. NIRN 
describes two major components that lead to 
effective implementation: Drivers and Stages. 

Drivers are specific components that inter- 
act with one another to promote change. 

The three main categories of drivers are Staff 

Competency, Organizational Supports, and 

Leadership. 

1. Staff Competency: Competency can be 
achieved through a combination of selec- 
tion, training, and coaching. 

a. Selection: Are you able to select indi- 
viduals more likely to embrace and 
become proficient in the intervention? 
One must start thinking about what cri- 
teria are used in interviewing, selecting, 
and promoting officers. While it may 
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be possible to begin hiring only those 
officers who have the skill you are look- 
ing for, it is likely that your criteria will 
change over the years. Additionally, once 
staff are hired it is often difficult, if not 
impossible, to terminate them, except 

in extreme circumstances. Thus, it may 
be more important to begin evaluating 
applicants for “coachability”—are they 
willing to receive feedback, and able 

to incorporate that feedback and do 
something different? In our district we 
began interviewing officer candidates 
for coachability by asking about spe- 
cific activities they had done recently to 
improve their skills, and their thoughts 
on audiotaping contacts for coaching 
and feedback. Such questions helped 
gauge candidates’ willingness to continue 
learning new ways of working. 


b. Training and Coaching: Research in a 


number of fields has consistently dem- 
onstrated that training will not lead to 
skill retention and regular use of the 
new skill on the job unless such training 
is followed by on-the-job coaching and 
feedback. For example, a study of teachers 
demonstrated that no teachers changed 
their behavior in the classroom follow- 
ing a training program that included 
lecture and demonstration by the trainer, 
and only 5 percent did so if they had to 
demonstrate the skill themselves in the 
training. In contrast, 95 percent of teach- 
ers used the new skill in their classroom 
if the training was followed by coaching 
and feedback in the classroom (Joyce and 
Showers, 2002). Similarly, research on 
Motivational Interviewing (MI) shows 
that most do not reach proficiency in MI 
if they do not receive follow-up coaching 
(Miller & Mount 2001). A study of MI 
with federal probation officers indicated 
that coaching by an expert trainer (ver- 
sus coaching by a peer) was significantly 
associated with officers becoming more 
proficient in MI as measured by the 
Motivational Treatment Integrity (MITI) 
coding system (Lowenkamp, 2011, per- 
sonal communication). 


2. Organizational Supports: 
a. Decision Data Support Systems: Data 


must be readily available to measure what 
you are doing. More important, the data 
must be reviewed regularly, and decisions 
must be made based on the data. For 
example, many districts have begun using 
a report that delineates the number of 
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contacts by risk (RPI score, see Figure 1). sheets, thereby increasing their aware- 


This report gives valuable feedback about 
how officers are spending their time 
and resources by risk level. This process 
of giving districts and officers feedback 
should encourage the evidence-based 
practice of seeing high-risk offenders 
more than low-risk offenders. In our 
district, we began by sharing the report 
with all officers and supervisors monthly. 
After seeing no change in contacts for a 
few months, we began requiring officers 
to include the RPI score on their field 


ness of the risk level of the offenders, 
and allowing supervisors a quick review 
of how officers were spending field time. 
Since that time, we have seen a significant 
change in contact levels; those scoring 
RPIs of 8-9 are now being seen more 
than twice as often as those scoring 0-1 
(see Figure 2). 


. Facilitative Administration: District 


leadership must do all it can to ensure 
that barriers to implementation are mini- 
mized. For example, do field contact 
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policies support focusing on high-risk 
clients, or are officers expected to see all 
clients within a geographic area, regard- 
less of risk level? Policies and procedures 
must be aligned with the new program 
or service, and feedback from the “front 
line” needs to be solicited and changes 
made to address issues that are raised. 
For example, in our district we regu- 
larly utilize anonymous surveys to solicit 
feedback from officers, and also have a 
Line Officer Committee whose sole pur- 
pose is to provide feedback and ideas to 
management. 

c. Systems Intervention: Districts must also 
work with collateral systems to ensure 
that they are on board with changes 
and support implementation efforts. For 
instance, educating the court, the U.S. 
Attorney's Office, and the Federal Public 
Defender’s Office is essential in order to 
avoid any barriers to implementation or 
to address barriers quickly (see Chapter 
6 of the Crime and Justice Institute's 
Implementing Evidence-Based Policy and 
Practice in Community Corrections, 2nd 
edition). 

3. Leadership: Strong, visible support is 
essential to conveying the message that 
the new program/practice is important 
and necessary. Chief Probation Officer 
Tom Tarr (New Hampshire) has modeled 
such leadership in his district's implemen- 
tation of the Strategic Training Aimed 
at Reducing Rearrest (STARR) program. 
Both he and his deputy chief have par- 
ticipated in every coaching call with their 


officers and continuously ensure that their . 


officers have the tools and resources neces- 

sary to fully implement STARR, such as 

bringing in ex-offenders to role play and 
providing additional training through local 
universities. 

Stages describes the various processes orga- 
nizations must go through to ensure effective 
implementation. They include Exploration, 
Installation, Initial Implementation, and Full 
Implementation. One of the first issues to 
acknowledge is the amount of time it will 
take to reach full implementation. Research 
consistently shows that implementation takes 
two to four years to complete (Fixsen, Naoom, 
Blase, Friedman, and Wallace, 2005). Issues to 
be addressed at each stage are outlined below. 

e Exploration stage: Management and 
staff must be given the time and oppor- 
tunity to fully explore the options 
available that will best fit the district's 


needs. At times this can be challenging, 
especially if the program/practice is a 
mandate that cannot be changed (such 
as the use of the Post Conviction Risk 
Assessment, PCRA). However, even in 
such situations it is important to talk 
through issues, as it allows staff time to 
“get ready” for change. Many are familiar 
with Prochaska and DiClemente’s Stages 
of Change (Prochaska et al., 1985) which 
suggests that individuals may be at differ- 
ent points of willingness to change, from 
those in Precontemplation (don’t see a 
need for change) to Action (those ready 
to change now). Prochaska, Prochaska, 
and Levesque (2001) suggest that a sig- 
nificant portion of individuals in an 
organization (up to 80 percent) may not 
be ready to change. The Exploration 
stage helps staff get “ready” for change 
by guiding discussion of what the change 
will mean for all involved. One useful 
tool to consider using is included in 
Figure 3, developed by NIRN. The tool 
helps staff discuss six broad areas that 
need to be considered when evaluating a 
new program or practice: 
1. The needs of the organization 
2. How the program/practice “fits” with 
other organizational issues 
3. What resources exist and will be 
needed 
4. The evidence for the program/ 
practice 
5. Others’ experience with 
implementation 
6. The organization’ ability to imple- 
ment the program/practice 


¢ Installation stage: During installation 


the district should begin preparing for 
implementation, which includes planning 
training, anticipating policy changes, set- 
ting up measurement tools, and identifying 
the broader district issues that may need 
to be addressed. For example, the new risk 
assessment is significantly longer than the 
current one in use, indicating a need to 
consider how the increased time will be 
handled. Our district looked at multiple 
ways to increase available time for the offi- 
cer, including more use of administrative 
or low-risk caseloads and revamping our 
intake procedure to make it more efficient. 
We also trained the supervisors on the 
tool first, and required them to complete 
at least two assessments, so they would 
fully understand the time and scoring 
issues involved with the new tool. Doing 


so allowed them to think through all the 
potential issues that might arise when we 
trained officers. We also experienced an 
unexpected benefit: several supervisors 
realized the new assessment was neither 
as difficult nor as time consuming as 
they expected, so they began talking it 
up to officers and informally exposing 
officers to the tool, such as explaining an 
item and potential scoring when staffing 
a case. They became “champions” for the 
tool before it was implemented! We also 
decided to delay our training of officers 
due to workload issues, which resulted in 
another benefit—we had a small group of 
officers who wanted to go ahead and train 
on the initially scheduled dates, who then 
also became champions as they realized 
that it was easier than they expected. The 
innovation diffusion literature calls these 
people the “early adopters” (Rogers, 1962) 
and suggests that these adopters can signif- 
icantly influence others to become involved 
in the innovation. These early adopters are 
also able to assist in streamlining imple- 
mentation, by identifying potential scoring 
issues and helping resolve those issues 
(such as clarifying score rules) before the 
training of all staff occurred. Working out 
such “bugs” helps ease the implementation 
for later adopters. 

Initial Implementation: This is where the 
work begins! Training starts and officers 
need to begin actually doing something 
different. Oftentimes, this is where the pro- 
cess can get stailed, as staff struggle with a 
new way of doing business. I’ve termed this 
time the “messy middle,’ and officers have 
described to me feeling anxious about their 
skills and feeling uncertain about what to 
do; they find themselves questioning all 
they do, and may at times feel paralyzed. 
Leadership is key during this difficult part 
of change. Leaders must reinforce that the 
new program/practice is worth doing, that 
the progress they are making is good (even 
though it may feel slow), and that you as a 
leader do not expect them to be competent 
in new skills yet. Leaders must realize that 
new skills take time and practice, and set 
realistic goals for skill attainment. For 
example, we first began teaching officers 
cognitive behavioral interventions (CBI) 
in October of 2010. By May 2011 many of 
our officers were routinely trying to use 
the intervention skill, and some are quite 
good at it. But they are not all fully com- 
petent in CBI, nor do I expect them to be. 
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implementation team comes in. Districts 
should be thoughtful in determining the 
members of a team; the team needs to under- 
stand not only the program/practice to be 
implemented, but also the research on effec- 
tive implementation. Unfortunately, most of 
our staff do not know this research. In order to 
overcome this obstacle, our district arranged 
for NIRN to be involved in our first imple- 
mentation team meeting for implementation 
of the risk assessment tool. We asked team 
members to read some basic information from 
NIRN before the meeting, then had NIRN 
staff consult as the team talked through issues. 
The NIRN staff member was able to point out 
to the team when their plans were consistent 
with the research on effective implementation 
and suggest alternatives when what they sug- 
gested was not in line with the research.' We 
then tasked the team with planning for needed 
structural/functional changes, devising a plan 
for coaching, officer support, and measure- 
ment of fidelity, and becoming the conduit for 
communication issues/barriers to the man- 
agement team. The implementation team has 


1 


NIRN staff recently provided similar train- 
ing to chief probation officers via a Webinar, and 
resources are available to members of federal pro- 
bation and pretrial services on the Administrative 
Office of the U.S. Court’s Evidence-Based Practice 
website. 


continuously revised the implementation plan 
in response to feedback. 

After all this work, and once you've 
achieved full implementation, you're finished, 
right? Wrong! The hallmark of EBP is that you 
continuously assess, evaluate, and improve 
what you do. Some call these “improve- 
ment cycles.” The message is the same—you 
should never be “satisfied” with where you 
are, but instead should be looking to continu- 
ously improve your organization's efficiency 
and effectiveness. In fact, research from the 
Washington State Institute for Public Policy 
(http://www.wsipp.wa.gov/) and others con- 
sistently shows that not doing interventions 
well will lead to no changes in recidivism, and 
potentially increases recidivism. Improvement 
cycles force the organization to continuously 
improve by actively studying the change and 
making adjustments as needed. 

In summary, implementing any change 
can easily fail unless districts follow evidence- 
based practices for effective implementation. 
By understanding the drivers and stages of 
implementation, districts can ensure that the 
infrastructure is in place to support implemen- 
tation and match organizational activities to 
guide the change process. Only in following the 
research can a district strategically implement 
changes that will endure for the long term. 
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THE PREDICTION OF RISK is ubiquitous 
in modern society (Beck, 1992). Physicians 
consider risk when treating patients, finan- 
ciers consider risk when making investments, 
and psychologists consider risk when work- 
ing with clients. Within the criminal justice 
system, predictions of risk guide discretion 
at all points (Gottfredson & Tonry, 1987). 
When police officers choose between formal 
citations and verbal warnings, they evaluate 
risk; when judges impose sentences upon 
defendants, they evaluate risk; and when 
community corrections officers monitor the 
conditions of pretrial defendants, parolees, 
and probationers, they, too, evaluate risk. 

Over time, research suggests, profession- 
als within the justice community develop 
the ability to distinguish high-risk offenders 
from those who present little risk of reoffend- 
ing (Fong, et al., 1990; Mossman, 1994). They 
do so by drawing upon their own personal 
experiences, using heuristics and other men- 
tal shortcuts to simplify complex calculations 
(Nisbett & Ross, 1980). But this kind of 
professional (or clinical) judgment is lim- 
ited to the experience of the decision maker 
and is subject to a host of faults: unreliable 
evaluations, discretionary decisions based 
upon biases and stereotypes, and politicized 
administration (Walker, 1993). An alterna- 
tive approach is to use statistically-derived 
instruments to predict actuarial risks of vio- 
lence, dangerousness, reoffending, rearrest, 
or reconviction. 
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The statistical prediction of recidivism 
risk has an 80-year history, and can be traced 
at least as far back as the 1928 parole predic- 
tion instrument developed by Ernest Burgess 
(Burgess, 1928). Early attempts to use actuarial 
risk assessment in the justice system were often 
controversial, particularly given high rates of 
false positives (Selective Incapacitation, 1982). 
Evaluators identifying subjects as dangerous 
were wrong twice as often as they were right 
(Monahan, 1981). Nevertheless, despite these 
flaws, research suggested that actuarial predic- 
tion outperformed the clinical judgment of 
even trained professionals across an array of 
disciplines (e.g., Meehl, 1954). The superior- 
ity of actuarial assessment over unstructured 
clinical judgment is a finding that has been 
replicated by many researchers (Grove & 
Meehl, 1996; Harris, 2006). One meta-anal- 
ysis of 136 studies concluded that statistical 
predictions were 10 percent more accurate 
than clinical judgments and were dramatically 
more accurate one third of the time (Grove, et 
al., 2060). The accuracy of assessment instru- 
ments also appears to have improved (Hilton, 
et al., 2006). A more recent meta-analysis of 
67 studies concluded that actuarial assess- 
ment generally is 13 percent more accurate 
than clinical judgment and is 17 percent more 
accurate in predictions of future violent or 
criminal behavior (Agisdottir, et al., 2006). 

Today, the academic debate is no 
longer about whetheractuarial assessments out- 

predict clinical judgments; that debate is 


long since over (Monahan, et al., 2001). Even 
the skeptics of actuarial risk prediction now 
acknowledge a consensus that actuarial judg- 
ments consistently outperform clinical ones 
(Harcourt, 2007; Litwack, 2001). Instead, the 
current debate is about whether there is any 
place in risk assessment for clinical judg- 
ment (Hanson, 2009). Some researchers argue 
for a synthetic approach, combining actu- 
arial and clinical techniques (e.g., Gottfredson 
& Moriarty, 2006; Sjéstedt & Grann, 2002, 
Sreenivasan, et al., 2000). After all, for all its 
strengths, actuarial prediction is not par- 
ticularly good at accounting for exceptional 
circumstances, predicting rare events, or pre- 
dicting risk for young people (for whom 
there is less historical information available) 
(Bullock, 2011). Other researchers, however, 
argue for an actuarial-only approach (e.g., 
Grove & Meehl, 1996; Quinsey, et al., 1998). 
They claim that the introduction of clinical 
judgment only reduces the accuracy of the 
instrument. And after all, “[e]ven if actuarial 
methods merely equal the accuracy of clinical 
approaches, they may save considerable time 
and expense” (Dawes, et al., 1989: 1673). 
Numerous commercial risk assessment 
instruments are available, all predicting 
recidivism about equally, all more accurate 
than unstructured clinical judgment (Yang, 
et al, 2010). Many jurisdictions use commer- 
cial instruments such as the PCL-R, CAIS, 
COMPAS, or the LSI-R. Other jurisdictions 
have adapted off-the-shelf instruments to fit 
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their specific needs or have developed their 
own in-house assessment tools. Used effec- 
tively, these assessment tools allow probation 
officers to accurately assess risk, a requisite 
first step in employing evidence-based prac- 
tices (Harris, 2006; VanBenschoten, 2008). 

Yet despite the lengthy history of statisti- 
cal risk assessment and despite a substantial 
body of research demonstrating that actuarial 
predictions outperform unstructured clinical 
judgment, probation officers—both in the 
United States and abroad—have exhibited 
skepticism, ambivalence, and outright hos- 
tility toward actuarial assessment devices. 
Irish probation officers have cultivated an 
attitude of “resistance” to assessment instru- 
ments (Fitzgibbon, et al., 2010). In England, 
Horsefield suggested that, using their clinical 
judgment, “it is not difficult for probation 
service staff to identify who is likely to com- 
mit further offences” (2003: 377), and argued 
that the real value of using actuarial risk 
instruments lies in justifying the operations 
within the probation service, competing for 
resources, and regulating staff behavior. In the 
United States, Schneider and her colleagues 
(1996) reported similar attitudes among 
Oklahoma probation officers. Officers held 
negative-to-neutral views about risk instru- 
ments (e.g., only 15 percent thought risk 
instruments are more accurate than officer 
judgment) but thought actuarial tools were 
useful in justifying supervision levels to the 
public and legislature. Lynch (1998) reported 
that California parole officers deliberately 
subverted directives issued by their actuarial 
risk managers. But even managers appear to 
express reservations about the value of risk 
assessment instruments. In a 2003 national 
survey of community corrections agencies, 
61 percent of respondents described them- 
selves as satisfied or very satisfied with the 
risk instruments used in their departments, 
but a full 39 percent described themselves 
as neutral, uncertain, or dissatisfied (Clem, 
2003: 22). 


Background and Research 
Question 


The tension between professional judgment 
and actuarial risk assessment affects the fed- 
eral probation and pretrial services system as 
well. Risk assessment is not new to the federal 
courts. The district court for the District of 
Columbia began using a risk prediction scale, 
the “U.S.D.C. 75” in 1970 (Hemple, et al., 
1976). This instrument was renamed the Risk 
Prediction Scale 80 (RPS 80) and adopted 


for use throughout the probation system in 
January of 1981 (Eaglin & Lombard, 1981). In 
September of 1997, the RPS 80 was replaced 
by the Risk Prediction Index (RPI), an eight- 
question, second-generation risk assessment 
tool (Lombard & Hooper, 1998). But many 
probation officers did not use the RPI scores 
they calculated (VanBenschoten, 2008), and 
did not always link supervision practices to 
risk levels (Lowenkamp, et al., 2006). 

Responding to the Criminal Law 
Committee's endorsement of evidence-based 
practices in the supervision of defendants 
and offenders (Judicial Conference, 2006), 
probation staff at the Administrative Office 
of the U.S. Courts have developed a new, 
fourth-generation risk assessment instrument, 
the Federal Post Conviction Risk Assessment 
(PCRA). The PCRA was validated on a large 
sample of federal probation cases (see article 
in this issue by James Johnson et al.). 

We were interested in whether use of the 
PCRA would improve the ability of federal 
probation officers to accurately assess risk. 
On the one hand, 50 years of research sug- 
gests that actuarial prediction consistently 
outperforms unstructured professional judg- 
ment (e.g., Agisdottir, et al., 2006; Grove & 
Meehl, 1996; Grove, et al., 2000; Monahan, et 
al., 2001); on the other hand, federal probation 
officers are considered to be the “creme de 
la creme” of community corrections officers 
(Buddress, 1997: 6). They are well educated, 
well trained, and often come to the federal 
system with substantial practical experience. 
Would the use of the PCRA allow even federal 
officers to improve their ability to assess risk? 


Methods 


The question of whether the use of the PCRA 
would improve the risk assessment skills 
of federal probation officers was investi- 
gated during four regional training meetings 
convened during 2010 and 2011. Federal 
probation officers from districts in the greater 
Washington, DC metropolitan region gath- 
ered in Washington, DC to participate in 
PCRA training; officers from districts in the 
eastern United States gathered in Charlotte, 
NC; officers from districts in the middle of the 
country gathered in Detroit, MI; and officers 
from districts in the western United States, 
including Pacific islands, gathered in Salt Lake 
City, UT. Approximately 150-350 officers 
attended each of the training meetings. 

Prior to the training session each offi- 
cer was asked to complete an eight-hour 
online training program that reviewed the 


fundamentals of risk, need, and responsivity 
(Andrews, et al., 1990). At each session, train- 
ers explained to the participating officers that 
they would be asked to assess an offender's 
risk based on a videotaped mock intake 
interview and supplementary written docu- 
mentation. Specifically, they were told that 
they would be asked to place the offender in 
the case vignette in one of four risk categories 
(low, low/moderate, moderate, or high) and to 
identify the offender's three most important 
criminogenic needs (in rank order). The 
description of the risk levels was not defined; 
thus the officers needed to define for them- 
selves what each risk level meant. Although 
the probation officers were in a large group 
setting, the trainers emphasized that officers 
were not to discuss their rankings of risk or 
identification of criminogenic needs until they 
submitted their data collection form. 

The case vignette consisted of a 24-minute 
mock intake interview (based upon an actual 
case, with identifiers and key case details 
modified in order to protect the offender's 
anonymity). The probation officer in the 
vignette asked the offender—a man in his fif- 
ties with a long history of methamphetamine 
addiction and firearms charges—a series of 
questions about the offender's criminal behav- 
ior, employment, social networks, cognitions, 
substance abuse, time in custody, and cur- 
rent accommodations. Supplemental written 
materials included a presentence report and 
release paperwork from the Federal Bureau 
of Prisons. 

The offender in the vignette was working 
and lived in a stable residence. He partici- 
pated in treatment, remained free of drug use, 
and could articulate a relapse prevention plan. 
He did not associate with anti-social peers and 
was in the process of developing a pro-social 
network. The correct score, according to the 
PCRA, was low/moderate risk. Specifically, 
the numerical score was 6 (PCRA scores range 
between 0 and 18). 

After the video concluded, officers were 
given as much time as needed to identify the 
risk level and three top criminogenic needs. 
Officers typically took between five and ten 
minutes to review the supplementary material 
and submit a complete data collection form. 
They were not provided with the correct score 
after this first exercise. 


On the second day of the training, after 
learning the scoring rules of the PCRA and 
practicing on several scenarios, probation 
officers viewed the training vignette for a sec- 
ond time. Instead of using their professional 
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judgment to identify the offender's risk level 
and criminogenic needs, they were asked to 
use the PCRA and identify a risk score. The 
officers were shown the same video and were 
provided with the same written supplemen- 
tary materials. Once again, they were asked to 
score the case independently and to provide 
their answers to the trainers. These actuarial 
(PCRA) risk assessments were collected and 
compared with the risk assessments made 
with clinical judgments. 

Risk category (low, low/moderate, moder- 
ate, or high) is an ordinal variable. As such, 
typical measures of central tendency and 
measures of dispersion do not apply. We were, 
however, interested in whether officers can 
accurately assign the offender into the correct 
risk category unaided by actuarial risk assess- 
ment and if the PCRA increases the reliability 
of the assessment of risk and thereby risk clas- 
sification. To evaluate the effect of the PCRA 
on reliability of risk assessments, we used the 
consensus measure (Tastle, et al., 2005) to 
measure dispersion. The consensus measure 
is a measure that ranges in value from 0 to 
1, with 1 representing complete agreement 
among those that ranked an item (in our case 
risk category) regardless of the category cho- 
sen. A value of 0 on the consensus measure 
(complete dissention) would be the result 
when two equal groups of participants rank a 
case at the far ends of the scale. This charac- 
teristic of the consensus measure is important, 
as it allowed us to determine whether the 
officers’ categorization of risk was consistent, 
regardless of whether or not their assessments 
agreed with the results of the PCRA. 


Results 


A total of 1,087 officers identified a risk cat- 
egory for the case vignette when asked to do 
so without administering the PCRA. A total 
of 1,049 officers provided a risk categorization 
for the case vignette using the PCRA. The dis- 
tributions of these ratings are presented in the 
following two figures. 

Figure | displays the frequencies (percent- 
ages are in parentheses) of risk categories 
identified by the officers using clinical judg- 
ment (without the use of the PCRA). As 
indicated in Figure 1, it is clear that the largest 
identified category of risk for the case vignette 
is moderate risk. Just over 50 percent of the 
officers indicated, based on the information 
provided, that the offender was moderate risk. 
Thirty percent of the officers identified the 
offender's risk level as low/moderate, while 
17 percent identified the risk level as high. A 
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much smaller percentage (2 percent) identi- 
fied the offender's risk level as low. Given this 
distribution of scores, a calculation of the con- 
sensus measure (Cns) yielded a value of 0.66. 
Figure 2 displays the distribution of risk 
categories assigned by officers when using the 
PCRA to guide their determination of risk. 
Note that in Figure 2, only three bars indicate 
the estimation of risk. No officers identified 
the offender's risk level as high when using the 
PCRA. A second noteworthy feature of Figure 
2 is that the largest category of risk identified 
by the officers accounts for ratings from 954, 
or 91 percent, of the officers. The consensus 
measure based on the distribution of these 


ratings yielded a Cns value of .93, or about 
1.4 times as great as the Cns measure yielded 
from the distribution of ratings in Figure 1. In 
addition, the officers selected the proper risk 
category, according to the PCRA, 91 percent 
of the time. Given that this was only these pro- 
bation officers’ first or second administration 
of the PCRA, these results are encouraging. 


Discussion 


Federal probation officers made more con- 
sistent and more accurate assessments of 
offender risk when using the PCRA than 
when using unstructured clinical judgment. 
Assessments made with the PCRA were more 
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accurate (e.g., more officers correctly identi- 
fied the risk level) and had greater consensus 
(e.g., even officers who did not correctly iden- 
tify the risk level selected categories adjacent 
to the actual risk level). These findings sup- 
port the view that, in assigning offenders to 
the correct risk category, actuarial prediction 
outperforms unstructured clinical judgment. 
Our findings are consistent with a robust body 
of work, collected over many decades (e.g., 
Agisdottir, et al., 2006; Grove & Meehl, 1996; 
Grove, et al., 2000; Meehl, 1954; Monahan, 
et al., 2001). But they are still remarkable. It 
is important to note that federal probation 
officers have to satisfy very high standards. 
They must meet medical standards, pass 
regular background investigations, possess at 
least a bachelor’s degree from an accredited 
university, and complete the six-week training 
program at the Federal Probation and Pretrial 
Services Training Academy. Typically these 
officers have prior probation experience from 
other jurisdictions. Additionally, these highly- 
skilled professionals are part of a single system 
with one set of national policies (with local 
variation) and a uniform training academy. 
Despite this, the federal probation officers 
produced a more consistent risk level assign- 
ment with the use of an actuarial tool. 

The research also indicates that clinical 
judgments tended to overestimate risk. It is not 
difficult to understand why. Ansbro notes that 
probation officers “face the mutually-exclusive 
targets of high accuracy and high throughput, 
and exist in a climate where failings in practice 
will be hunted for if an offender commits a 
serious offence whilst on supervision” (2010: 
266). A signal detection analysis lies beyond 
the scope of this article, but in a situation 
where there are dire consequences to missing 
a true positive (i.e., not identifying a high-risk 
offender as such) and few direct costs to offi- 
cers when making false positives (i.e., wrongly 
identifying a low-risk offender as high-risk), it 
is easy to see why officers would yield to the 
so-called precautionary principle identified by 
Kemshall (1998). Of course, over-supervising 
low-risk offenders is expensive, and diverts 
resources away from the high-risk offenders 
who need them. Austin analogizes this to a 
“hospital that decides to provide intensive 
care for patients who have a cold—the treat- 
ment is not only unnecessary but expensive” 
(2006: 63). There is also research suggesting 
that over-supervising low-risk offenders can 
make them worse, affirmatively increasing 
their likelihood of recidivism (Lowenkamp & 
Latessa, 2004). Actuarial tools can serve as a 


valuable check against the precautionary prin- 
ciple. They can provide a means of engaging in 
professional triage, ensuring that resources are 
allocated where they should be, maximizing 
community safety while allowing for offender 
rehabilitation (Flores, et al., 2006). 

It is also important to note that specific 
descriptions of the risk terms were not pro- 
vided for the officers. This may have caused 
some of the risk category assignment varia- 
tion. What “low risk” means to two different 
officers may vary. This means that the varia- 
tion in risk assignment may be due to how the 
case is seen and understood by an officer, but 
equally concerning is that the difference may 
also be due to various definitions of language 
that officers and national policy use related 
to risk. 

In a landmark article, Feeley and Simon 
suggested that the rise of risk assessment 
was symptomatic of a shift to a new penol- 
ogy: “[T]he new penology is markedly less 
concerned with responsibility, fault, moral 
sensibility, diagnosis or intervention and treat- 
ment of the individual offender. Rather, it 
is concerned with techniques to identify, 
classify, and manage groupings sorted by 
dangerousness. The task is managerial, not 
transformative” (1992: 452). Without ques- 
tion, the use of risk assessment instruments 
in community corrections has exploded since 
Feeley and Simon published their article, and 
its ascendance has been criticized by many 
thoughtful critics (e.g., Hannah-Moffat, et 
al., 2009; O'Malley, 2004; Wandall, 2006). 
Indeed, Harcourt (2007) demonstrates that 
risk-based justice may actually increase the 


overall amount of crime in society. In jurisdic- 


tions around the world, probation and parole 
officers have resisted the tyranny of risk and 
rejected managers’ instructions to manage 
offenders under their supervision by risk 
score (Fitzgibbon, et al., 2010; Lynch, 1998). 
But this view of risk assessment may be too 
dystopian. Other commentators have realized 
that the consequences of risk assessment are 
far more nuanced than its critics suggest. For 
example, Robinson (2002) notes that actuari- 
alism’s focus on outcomes actually underlies 
the new rehabilitation of “what works” (see 
Petersilia, 2004; Taxman, et al., 2004). To be 
sure, this is a form of rehabilitation that takes 
public safety as its ultimate object—not the 
transformation of every individual offender 
(Robinson, 2002). But instead of contributing 
to an inexorable increase in prison popu- 
lations and persons under supervision—a 
population that exceeded five million, or 1 


in 31 US. citizens, during 2009 (Pew Center, 
2009)—risk assessment can reduce prison and 
community corrections populations (Bonta, 
2008). By operating as a check against the 
precautionary principle and reducing over- 
classification, actuarial risk assessment can 
reduce recidivism among low-risk offenders 
by ensuring that they are not over-super- 
vised. It can simultaneously reduce recidivism 
among high-risk offenders by ensuring that 
these individuals are carefully supervised 
and provided with interventions that corre- 
spond to their criminogenic needs. Instead of 
stripping the humanity from probation work 
(Wandall, 2006), actuarial risk assessment 
with the PCRA can allow federal probation 
officers to be far more effective in facilitat- 
ing real transformative change in the lives of 
offenders. 
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COMMUNITY SUPERVISION IS one of 
the most widely imposed court responses, 
with approximately 5,095,200 or 70 percent of 
the correctional population being under com- 
munity supervision (Glaze, 2010). Despite its 
popularity, researchers have limited insight 
into whether community supervision is an 
effective strategy for reducing recidivism. 
The most recent reviews of the effective- 
ness of community supervision (Solomon, 
Kachnowski, & Bhati, 2005; Aos, Miller, & 
Drake, 2006; Bonta, Rugge, Scott, Bourgon, & 
Yessine, 2008; Green & Wink, 2010) brought 
sobering results. More than three decades 
after Martinson (1974) summarized the find- 
ings of his review of rehabilitation efforts by 
saying “with few and isolated exceptions, the 
rehabilitative efforts that have been reported 
so far have had no appreciable effect on 
recidivism,’ Bonta and his colleagues (2008) 
have found that more recent research yields 
no better results: the impact of community 
supervision is limited at best and non-existent 
in the most pessimistic interpretation. 

The Urban Institute (Solomon, et al., 2005) 
reached a similar conclusion after analyzing 
the impact of post-prison supervision on 
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re-arrest outcomes. The researchers, after 
comparing mandatory parolees with simi- 
lar prisoners released without supervision, 
concluded that “overall, parole supervision 
has little effect on re-arrest rates of released 
prisoners.” Consistent with the findings of 
the Urban Institute, Green & Wink (2010) 
declared “... probation does not alter the prob- 
ability of recidivism” after tracking more than 
1,000 offenders randomly assigned to nine 
judicial calendars. Simply put, the recidivism 
rate of those placed on probation was no dif- 
ferent from that of those who weren't placed 
on probation. Taxman (2002) further affirms 
the notion of ineffectiveness after reviewing 
studies on intensive supervision and caseload 


- size. Taxman states that “unless the contacts 


are more than check-ins it is unlikely that they 
will impact outcomes.” 

Collectively, these reviews indicate that com- 
munity supervision has little to no impact on 
the likelihood of future crime. However, none 
of these studies examined exactly what occurs 
in meetings between officers and those on 
supervision. To answer this question, research- 
ers and practitioners have started to unpack the 
“black box” of supervision (Bonta et al., 2008) to 
determine what might impact client outcomes. 
Results suggest that the core of community 
supervision must be built on a foundation 
that targets those at highest risk of engaging in 
criminal behavior, that the areas targeted are 
those closely linked to future criminal behavior, 
that barriers to treatment must be removed, and 
that cognitive-behavioral strategies must be uti- 
lized (Andrews, Zinger, Hoge, Bonta, Gendreau, 
& Cullen, 1990). Likewise, research supports 
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the notion that the quality and nature of the 
relationship between the client and the supervi- 
sion officer has an impact on outcomes (Skeem 
et al., 2007; Paparozzi and Gendreau, 2005). 
Finally, Jalbert and colleagues (2011) recently 
completed an analysis of the impact of caseload 
size on supervision outcomes, and concluded 
that caseload size can improve outcomes only if 
used in combination with the effective control- 
ling and correctional strategies described above. 

Moving from a “check-in” to the use of 
a core skill set to increase effectiveness is 
supported by Andrews and Kiessling (1980), 
Dowden and Andrews (2004), Trotter (1999), 
and Taxman (2008). Andrews and Kiessling 
(1980) introduced the five dimensions of 
effective correctional practice that were 
designed to enhance the potential of rehabili- 
tation programs for offenders. Dowden and 
Andrews (2004) provided a meta-analytic 
review of the core correctional practices indi- 
cating that the use of authority, disapproval, 
reinforcement, modeling, teaching problem 
solving skills, and structured learning are all 
related to the effectiveness of correctional ser- 
vices. While much of the research reviewed by 
Dowden and Andrews focused on treatment 
programs, the Bonta et al. (2010) findings 
are consistent with other research focused 
on testing the use of these skills in a com- 
munity supervision setting (Trotter, 1996 & 
1999; Taxman et al., 2006). Bonta et al. (2010) 
affirms the relationship between specific core 
correctional skills and the effectiveness of 
community supervision officers, noting that 
those officers trained in the skills utilized 
the skills more often and clients of officers. 
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trained in the skills had lower recidivism rates 
than those of untrained officers. 

The existing research (Bonta et al., 2008; 
Taxman et al., 2006; and Trotter, 1996) is 
encouraging and points to a need for fur- 
ther research on the training of community 
supervision officers providing direct service 
to clients. The current study uses a larger 
sample size than previous studies to further 
investigate the application of techniques that 
influence change and provide a model for 
targeting dynamic risk factors. The current 
study also uses random assignment to control 
for the selection and assignment biases associ- 
ated with observational studies and the use of 
volunteer participants. 


Method 


This study used an experimental pre-post test 
design. All officers who volunteered for the 
training and study were randomly assigned 
to the experimental (trained) and control 
(untrained) groups.' Random assignment was 
completed such that 66 percent of the officers 
were randomly assigned to the experimental 
condition and the remainder assigned to the 
control condition. While officers were, for 
the most part, randomly assigned to the two 
conditions in this study, the clients were not. 
However, as is the case with most community 
supervision agencies, once geography was 
factored in, the client assignment process was 
based on rotation, caseload size, or some other 
factor unrelated to officer status in this study. 


Participants 


Officers 


The study began with 53 officers in the experi- 
mental group and 35 in the control group. 
Immediately following the training, 6 officers 
were lost from one district (both experimental 
and control group) due to a lack of desire to 
participate. This reduced the numbers to 49 
and 33. An additional group of officers (8 from 
the experimental group and 7 from the control 
group) were lost because some officers received 
promotions, left their position with the judi- 
ciary, dropped out of the study, or did not have 
moderate- and high-risk clients both pre- & 
post-training. This reduced the total number 
of officers in the experimental group to 41 and 


1 


Two districts would not allow their officers to be 
randomly assigned. The analysis reported in this 
article included all the officers. Analysis with and 
without those officers was conducted and revealed 
only slight differences in the findings. These differ- 
ences do not impact the overall trends reported or 
the conclusions of this article. 


the total number of officers in the control group 
to 26. The attrition rate of 18 percent, while of 
some concern, retained the original distribu- 
tion of officers across the two groups. 


Clients 


Clients were identified for inclusion in this 
study based on when their period of supervi- 
sion began. While clients were not randomly 
assigned to experimental and control group 
officers, clients were assigned to officers inde- 
pendent of the officers’ STARR training status. 
The pretrial sample is made up of cases that 
were assigned to pretrial supervision. Pre- 
training cases were those cases that began 
pretrial supervision during 2007 and 2008 and 
terminated supervision before the training 
event date. Post-training pretrial cases were 
those cases that were assigned to study officers 
after May 31, 2009. Post-conviction pre-train- 
ing cases were identified as those cases that 
began their supervision between May 31, 2007 
and May 31, 2008, as this allowed for a follow- 
up time of at least 12 months. Post-conviction 
post-training cases were those cases assigned 
for supervision after May 31, 2009 up until 
December 12, 2009.’ 

In all there were 345 pre-training cases 
assigned to control officers and 446 pre-train- 
ing cases assigned to the experimental officers. 
A total of 218 post-training cases were assigned 
to the control officers and 295 post-train- 
ing cases were assigned to the experimental 


Criminal record checks for the post-conviction 
sample were run on December 21, 2010. 


officers. A breakdown of these cases by pretrial 
versus post-conviction supervision is provided 
in Table 1. As indicated in Table 1, a much 
larger number of the clients included in this 
study are those on post-conviction supervision. 

The demographic statistics of the clients 
included in this study are presented in Table 
2. Fifteen percent of the sample is female and 
57 percent belongs to a minority race or eth- 
nicity. Half of the clients were moderate-risk 
and half were high-risk according to the RPI.’ 
The average age for the clients included in this 
study is approximately 35. Table 2 also shows 
the descriptive statistics by group (experi- 
mental versus control). None of the observed 
differences were significant at the p < .05 
level. Analysis of demographic characteristics 
by pre- and post-status and pre-post status 
by group (pre-training control group, post- 
training control, pre-training experimental, 
and post-training experimental) revealed no 
statistically significant differences. 


Procedures 


This training was intentionally designed to be 
responsive to the literature on technology trans- 
fer‘ and the use of one-on-one officer-client 
interactions to reduce risk and thereby client 


3 


The Risk Prediction Index (RPI) was introduced 
in 1997. The RPI uses 8 mostly static items to pre- 
dict the likelihood of re-arrest while on supervision 
(Lombard, Hooper, Rauma, 2001). 


* The research on diffusion of innovation and 


implementation is voluminous. Interested readers 
should see Rogers, 2003 and Fixsen, Naoom, Blasé, 
Friedman, & Wallace, 2005. 


TABLE 1. 
Distribution of pre- and post-training cases by supervision type 
Pre Training Post Training 
Experimental Control Experimental Control 
Pretrial ae 56 30 26 
Post conviction 394 289 265 192 
TABLE 2. 
Descriptive statistics for demographic characteristics of clients 
All (1304) Experimental (747) Control (547) 
Variable Percent or Mean Percent or Mean Percent or Mean 
Female 15% 15% 15% 
Minority 57% 57% 57% 
Percent moderate-risk 50% 48% 53% 
Percent high-risk 50% 52% 47% 
Age 34.78 34.95 34.55 
RPI Score 5.61 5.69 5.49 
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recidivism. Officers in the experimental group 
participated in a 3% day classroom training that 
included a discussion of the theory supporting 
the development of the STARR curriculum, a 
demonstration of each skill, exercises, and an 
opportunity for officers to practice each skill 
and receive feedback. The theory discussion 
reviewed the risk, need, responsivity model 
(see Andrews & Bonta, 2003) and the research 
demonstrating the effectiveness of a skill- 
focused supervision approach. The STARR 
skills themselves include specific strategies for 
Active Listening, Role Clarification, Effective 
Use of Authority, Effective Disapproval, 
Effective Reinforcement, Effective Punishment, 
Problem Solving, and Teaching, Applying, and 
Reviewing the Cognitive Model. For each 
strategy skill cards were developed that outline 
the specific activities officers needed to do to 
successfully deliver the strategy. A fundamen- 
tal focus for each skill is the internalization of 
strategies so that defendants/offenders begin 
to learn and apply the strategies on their own. 
In addition to the skill cards, video examples 
of some skills were presented, while others 
were demonstrated live. The exercises allowed 
officers to practice each skill. For example, after 
listening to a discussion about reinforcement, 
officers were asked to identify a behavior and 
a reinforcement strategy for a specific offender, 
then role play that interaction with another 
officer. The officers (experimental and control) 
were asked to send in audiotaped interactions 
(1 before the training event and up to 30 after 
the training event) at designated intervals: ini- 
tial meeting with the client, an interaction with 
the client 3 months later, and then a third and 
final taping 3 months after that (6 month inter- 
action). Officers made recordings with up to 10 
moderate- or high-risk clients. The audiotapes 
were used to gain a better understanding of skill 
development and provide feedback to the offi- 
cers. Four “booster” trainings’ were held over 
the next year to provide officers with additional 
training on skill deficits identified on the tapes. 
Booster trainings were delivered by phone and 
included discussion of specific skills, audiotape 
examples of the skill, and individual feedback 


and coaching. 


Measures 


Intermediate officer measures 

Officer’s use of the skills taught during the 
training was measured by reviewing audio- 
tape recordings of interactions with clients 


Participation in the booster training events was 
voluntary. 


recruited for the study. In all 731 audio 
recordings were submitted for review. This 
included 491 recordings from the experimen- 
tal group and 240 from the control group. 
The audiotapes were coded by trained raters 
who focused primarily on behaviors con- 
sistent with the skills introduced during the 
training. For example, with reinforcement or 
disapproval, raters coded whether the officer 
identified the specific behavior and whether 
the officer had the offender explore the short- 
and long-term consequences of the behavior. 
For the cognitive model, raters coded whether 
the model was taught, applied, or reviewed, 
and coded for discussions of internal cues, 
consequences of internal cues, and identifica- 
tion of counter thoughts. Finally, raters coded 
what topics were discussed in the interaction. 


Client Outcomes 


Two different outcome measures were used 
in this study. For pretrial clients, the out- 
come measure was failure on supervision, 
as evidenced by failure to appear in court, 
supervision being revoked, or being arrested 
for a new criminal charge while on pretrial 
supervision. The data for this measure was 
taken from PACTS.° The outcome measure for 
the post-conviction cases was arrest for new 
criminal behavior, as identified in the NCIC 
or ATLAS databases.’ 

For clients on post-conviction supervision, 
the follow-up time was standardized to 12 
months. For pretrial clients the time period 
was limited to the time they were on pretrial 
release. For the pre-training group, the average 
time on pretrial release was 229 days and the 


post-training average time on pretrial was 185 


days (observed differences between experi- 
mental and control groups and across time 
periods within groups were not significant). 


® PACTS (Probation/Pretrial Services Automated 
Case Tracking System) is an electronic case man- 
agement tool used by probation and pretrial 
services officers in all 94 federal districts to track 
federal defendants and offenders. At the end of each 
month, districts submit case data into a national 
repository that is accessible to the Administrative 
Office of the U.S. Courts (AO), Office of Probation 
and Pretrial Services. 


Data on arrests were gathered from record 
checks that include a search of the National 
Crime Information Center and the National Law 
Enforcement Telecommunications System which, 
together, include data on federal and local charges. 


Analysis 


Bivariate analyses were used to assess the 
change in officer behavior from -pre- to post- 
training and across the experimental and 
control groups and to assess the impact of 
the training on client outcomes. Multivariate 
analyses were used to determine the interac- 
tion between individual client characteristics 
and officer training and their impact on 
client outcomes. 


Results and Discussion 


Intermediate Outcomes 


The 88° officers participating in the study sub- 
mitted 598 audio recordings for review. This 
included 400 from the experimental group 
and 198 from the control group. The audio- 
tapes were coded by trained raters using a 
structured guide primarily focused on behav- 
iors supported by core correctional practices. 
This analysis focuses on three intermediate 
variables: a) the officer’s use of reinforcement 
and disapproval, b) interactions where cogni- 
tions, peers, or coping skills were discussed, 
and c) the officer’s use of cognitive techniques 
during interactions with clients. Analysis of 
pre-training audiotapes showed no difference 
between the experimental and control groups 
in the use of these skills. 

As seen in Table 3, officers in the experimen- 
tal group used reinforcement and disapproval 
at nearly twice the rate of untrained officers. 
This suggests that trained officers were almost 
twice as likely to capitalize on opportunities 
to use behavioral strategies that help shape 
client behavior. The finding provides cause for 
optimism because of the demonstrated impact 
of operant conditioning techniques like rein- 
forcement and disapproval (see Dowden and 
Andrews 2004). 

Cognitions, peers, and impulsivity empiri- 
cally represent some of the strongest predictors 
of future criminal behavior. A primary focus of 
STARR is addressing dynamic risk factors using 
a structured cognitive-behavioral approach. 
Table 4 shows post-training interactions where 
cognitions, peers, and impulsivity were dis- 
cussed. As seen in Table 4, discussions about 
cognitions, peers, and impulsivity were signifi- 
cantly more likely to occur among officers in 
the experimental group than among officers of 
the control group (44 percent vs. 30 percent). 
This represents a significant difference in how 
often primary risk factors are targeted. 


’ The study began with 88 officers and asked each 
of them to submit a taped interaction pre-training. 
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TABLE 3. 
Post training use of reinforcement and disapproval by group 
Did Not Use Skill Used Skill | 
N N % 
Experimental 230 66% 117 34% 
Control 139 83% 29 7% 
X(1) = 15.090; p = 0.05; n = 0.171 
TABLE 4. 
Post-training percent of interactions where cognitions, peers, 
or impulsivity were discussed 
Not Discussed Discussed 
N N % 
Experimental 196 56% 151 44% 
Control 70% 30% 
X°(1) = 8.222; p s 0.05; n = 0.126 
TABLE 5. 
Post-training percent of interactions where cognitive model was used 
Not Discussed Discussed 
N N % 
Experimental 289 83% 58 17% 
Control 167 99% 1 % 


X'(1) = 28.995; p s 0.05; n = 0.237 


TABLE 6. 


Failure rates for moderate and high-risk clients by group membership 


Failure Rate 


Pre-training' 


Post-training? 


Control Officers 


130/345 (38%) 


75/218 (34%) 


Experimental Officers 


175/446 (39%) 


(1}= 0.199; p > 0.05 
2 ¥? (1)= 3.798; p < 0.05 


In addition to targeting dynamic risk factors 
and using operant conditioning techniques, 
analysis suggests significant differences in 
officers’ use of the cognitive model. As seen in 
Table 5, control group officers used the cogni- 
tive techniques in 1 percent of interactions 
where the skills were applicable compared to 
17 percent by experimental group officers. 
This suggests that experimental group officers 
were significantly more likely to use the cog- 
nitive techniques to teach offenders the link 
between thinking and behavior. 


Client Outcome 


Our first analysis aimed at identifying the 
difference in failure rates for those clients 


assigned to two groups of officers prior to 
the training. The first panel of Table 6 indi- 
cates that there was no difference in client 
failure rates between the groups prior to 
STARR training; control group officers had 
a failure rate of 38 percent, whereas clients 
assigned to the experimental officers had a 
39 percent failure rate. The second panel of 
Table 6 displays the post-training failure rates 
of the clients based on group assignment. 
The control group cases had a post-training 
failure rate of 34 percent, which did not signif- 
icantly differ from the pre-training failure rate. 
The post-training failure rate for the clients 
assigned to experimental group officers was 
26 percent, which is significantly lower than 


the pre-training experimental failure rate, 
and more importantly, than the post-training 
failure rate for the control group (see note 2 
of Table 6). 

Taken together, these results indicate that 
training can significantly impact strategies 
used by officers during supervision, and that 
these strategies lead to lower failure rates. The 
overall difference in failure rates between the 
trained and untrained groups is nine percent- 
age points, which equates to a reduction in 
relative-risk of approximately 25 percent and 
is quite remarkable, as these differences were 
achieved with approximately 40 total hours of 
training, no reductions in caseloads, and no 
additional work hours from officers. 

The next set of analyses focused on inves- 
tigating the impacts of group membership on 
client outcomes by risk, specifically whether 
changes and differences in failure rates dif- 
fered drastically between moderate- and 
high-risk clients. Tables 7 and 8 present the 
failure rates by group for moderate- and high- 
risk clients. 

Table 7 presents the failure rates by group 
for the moderate-risk clients. In the first 
panel note that the failure rates for the control 
and experimental groups pre-training were 
again statistically indistinguishable from one 
another (31 percent and 32 percent respec- 
tively). The failure rates for the control group 
pre- and post-training and the experimental 
group pre-training failure rate also do not 
differ significantly from one another. Post- 
training failure rates between the groups 
indicate a very different trend. The post- 
training failure rate for the control group is 32 
percent, while the post-training failure rate for 
the experimental group is 16 percent. This is 
an absolute reduction of 16 percent and a rela- 
tive risk reduction of 50 percent. This again 
is noteworthy given the amount of training 
the officers were given and the fact that the 
experimental group officers had no additional 
resources or reduced caseloads. 

The failure rates by group for the high- 
risk clients are contained in Table 8. The first 
panel of Table 8 indicates, once again, that 
the pre-training failure rates across the two 
groups did not differ significantly from one 
another. The second panel of Table 8, which 
displays the post-training failure rates across 
the two groups, also indicates no difference. 
An important trend that occurs for both 
groups is the decrease in failure rates from 
pre- to post-training. This is likely the result 
of other efforts that had been ongoing in the 


| 78/295 (26%) 
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TABLE 7. 


Failure rates for moderate-risk clients by group membership 


Failure Rate 


Pre-training' 


Post-training’ 


Control Group 
Officers 


58/188 (31%) 


36/113 (32%) 


Experimental 
Group Officers 


71/221 (32%) 


22/135 (16%) 


"2 (1)= 0.782; p > 0.05 
2 y2 (1)= 8.314; p < 0.05 


TABLE 8. 


Failure rates for high-risk clients by group membership 


Failure Rate 


Post-training” 


39/105 (37%) 


Pre-training' 
Control Group 72/157 (46%) 
Officers 
Experimental 104/225 
Group Officers (46%) 


56/160 (35%) 


'? (1)= 0.005; p > 0.05 
> x2 (1)= 0.127; p > 0.05 


TABLE 9. 
Multivariate analyses of client outcomes 
Model 1 Model 2 Model 3 

Variable B SE P B SE P B SE P 
Minority .088 .126 485 .085 501 .088 486 
Female -.435 .182 OFF -.44] 182 015 43 .016 
Age -.040: -.040 .007 .000 040 007 
High-risk .OOO 556 .000 467 
Pre-post -.423 -.142 .187 448 139 .187 455 
Period 
Group 110 365 .760 .620 .080 152 .601 
Pretrial .156 192 415 -.149 .192 603 157 192 413 
Group x -.505 .044 810 326 013 
Pre-Post 
Pre-Post X 
High Risk 
Constant 719 .276 .009 .628 281 025 672 .282 017 


districts selected for this study.’ At any rate, it 
doesn't appear that the STARR skills, in this 
context, produced any beneficial results over 
and above the targeted efforts of the officers 
in the study. 

The bivariate analyses indicated that the 
two groups of clients did not differ in terms of 


” All of the districts included in this study were 
part of the Research-2-Results project, which pro- 
vided funding for districts to use assessment and 
services for higher-risk offenders. 


race, age, gender, or time at risk. Intra-group 
pre-post differences in race, age, gender, and 
time at risk were determined to be statistically 
not significant. Even so, since clients are not 
necessarily randomly assigned to officers, we 
felt it was important to construct and estimate 
a series of multivariate logistic regression 
models predicting client failure. The three 
models are presented in Table 9. The only 
difference across the three models is the addi- 
tion of interaction terms. More specifically, 


Model 2 includes an interaction term between 
group membership and pre-post time period 
which isolates the effects for clients assigned 
to experimental officers after the training. 
Model 3 includes an interaction term between 
group membership, pre-post time period, and 
high-risk status. Thus the additional interac- 
tion term allows us to better understand the 
effects for moderate- and high-risk clients net 
the effects of other variables in the model. 

The three multivariate models presented in 
Table 9 all predict the same outcome and use 
the same set of control variables. The differ- 
ence between the three models is the addition 
of the interaction terms as described above. 
In each of the three models the parameter 
estimate for minority status is not signifi- 
cant, meaning that once the other factors are 
controlled for minority status is not associ- 
ated with failure. In all three models female 
is inversely associated with failure, age is 
inversely associated with failure, and high-risk 
status is positively associated with failure. In 
each of the three models the parameter esti- 
mate for group, which captured whether an 
offender was assigned to a control or experi- 
mental group officer, was not significant. 

The parameter estimate for the pre-post 
period (coded as | for post-training period) 
is significant and negative in Model 1. Once, 
however, the interaction term between 
group membership and pre-post period is 
introduced, the parameter estimate for the 
pre-post variable is no longer significant. 
This indicates that what was driving the effect 
of the pre-post period in Model 1 is attribut- 
able to the reductions in the post-training 
experimental group. Model 3 introduces an 
interaction term that quantifies the impact of 
high-risk clients in the post-training experi- 
mental group. As indicated, compared to 
moderate-risk clients, high-risk clients in the 
post-training experimental group are slightly 
more likely to fail; however, this parameter 
estimate does not quite reach the typically 
accepted probability values associated with 
statistically significant findings. 

In converting the log-odds ratios into 
probabilities, we are able to develop a better 
understanding of a particular variable while 
holding the other factors constant. In doing 
so, we estimated the probability of failure for 
a 35-year-old, minority, moderate-risk male 
who was on post-conviction supervision and 
in the control group prior to training to be 
at .35. A 35-year-old, minority, moderate- 
risk male that was in the experimental group 
prior to training has an estimated probabil- 
ity of failure at .36. A case with those same 
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characteristics that was on an experimental 
officer's caseload after training had a prob- 
ability of failure at .18, while a case with the 
same characteristics on a control officer's 
caseload after training had a .31 probability 
of failure. One can quickly see from these 
calculated numbers that the cases on the 
experimental officers’ caseload had about half 
the failure rate. This decrease is not as large 
for high-risk cases; however, it is still in the 
expected direction and the parameter estimate 
that captures the post-training difference in 
effect for the high-risk cases was positive 
(indicating an increase in the probability of 
failure and thereby a decrease in effect) but 
not statistically significant. A visual display of 
the predicted probabilities of re-arrest by risk 
and group assignment is provided in Figure 
1 below. 


CONCLUSION 


This study used an experimental design to 
assign officers to an experimental and control 
condition focused on training officers in spe- 
cific strategies to use during direct supervision 
of defendants/offenders. Clients supervised by 
these officers both before and after the training 
were used to assess the impact of the training 
on officer behaviors and client outcomes. 
Results indicate that officers utilized effective 
strategies more often post training, and that 
client outcomes were impacted. Clients super- 
vised by the experimental group of officers 
after the training had far superior outcomes, 
even after controlling for individual client 
level characteristics. A sub-analysis indicated 
that the effects of the experimental condi- 
tion were not present for high-risk clients. 


Nonetheless, this study, a fairly rigorous test, 
indicated that training in STARR is associated 
with 50 percent reductions in 12-month fail- 
ure rates for moderate-risk clients. 

There are a number of limitations to this 
study. First, the clients were not randomly 
assigned to the officers. The various districts 
involved in this study confirmed that client 
assignment was based on a combination of 
geography, case specialty (substance abuse 
specialists received a higher percentage of 
these types of clients), and attempts to balance 
caseload sizes. However, while the clients do 
not appear to differ in terms of individual level 
characteristics across the groups or pre-post 
time periods, it could be the case that some 
unmeasured bias in assignment exists. Second, 
a full analysis of the intermediate measures 
and their relation to client outcomes has not 
yet been conducted. These analyses might 
lead to important adaptations in the training 
and implementation. Third, while there was 
an attempt to minimize attrition, we did end 
up losing just under 20 percent of the officers 
that started this study. Fourth, the study was 
conducted by those that developed the train- 
ing program and trained the officers. Given 
previous findings on the relationship between 
demonstration studies and real-world applica- 
tions, independent and external replications of 
this study will be important. 

Notwithstanding, there are some very 
important policy implications to take from 
this study. First, it appears that officers can be 
trained in behaviorally based skills and they 
can use those in their one-on-one interactions 
with offenders. The data presented on the 
intermediate measures provides support for 


FIGURE 1. 5 
Predicted pre-post training failure rates by group and risk 
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this conclusion. More important, the bivari- 
ate and multivariate analyses of outcome 
measures supports the conclusion that train- 
ing probation officers in behavioral strategies 
might have a profound effect on failure rates of 
clients in the correctional system. Subsequent 
research should focus on continuing to test 
this conclusion. 
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IN RECENT YEARS, there has been growing 
interest on the part of federal judges, proba- 
tion officers, and others in establishing reentry 
court programs to manage the reintegration 
of offenders from prison to the community. 
These programs generally incorporate fea- 
tures of drug courts that have been shown to 
reduce recidivism in state and local jurisdic- 
tions. They employ the authority of the court 
to impose graduated sanctions and positive 
reinforcements in a team approach typically 
involving a judge, probation officer, assistant 
US. attorney, assistant federal defender, and 
contract services provider. 

Because the reentry court movement at 
both the state and federal level is still in its 
infancy, there is little empirical research on 
whether these programs effectively reduce 
recidivism. This paper summarizes several 
studies of federal reentry court programs. It 
first describes the history of drug courts and 
reentry courts in the states. It then provides 
a brief overview of existing federal reen- 
try court programs and summarizes recent 
evaluations in three federal districts: Oregon, 
Massachusetts, and the Western District of 
Michigan. Finally, it discusses the studies 
findings, limitations, and implications. 


Background of State Drug Court 
and Reentry Court Programs 


Drug courts, which have become widespread 
in the states since their introduction more 
than two decades ago, are specialized courts 
designed to handle cases involving nonvio- 
lent, substance-abusing offenders through 
a comprehensive program of supervision, 


drug and 
immediate sanctions and incentives. They 


transform the adversarial role of the court into 


testing, treatment services, 


a non-adversarial forum for problem-solving 
collaboration among the judiciary, prosecu- 
tion, defense bar, probation, law enforcement, 
and treatment services agencies (Drug Court 
Professionals, 1997; Department of Justice, 
2006). Depending on the structure of the drug 
court, successful completion may be accom- 
panied by dropping the charges (pre-plea/ 
diversionary court) or expunging the offense 
from the record (post-plea court). The avail- 
able research suggests that state drug courts 
have succeeded in reducing recidivism and 
that more study is needed to identify what 
aspects make them effective (Aos, Miller, 
and Drake, 2006; Latimer, Morton-Bourgon 
and Chretien, 2006; Wilson, Mitchell and 
MacKenzie, 2006; Lowenkamp, Holsinger and 
Latessa, 2005). 

The principles and processes of drug 
courts have been adopted by “problem- 


solving courts” to address other forms of 


chronic behavior by defendants.' Examples 
of problem-solving courts include mental 
health courts, domestic violence courts, 
homeless courts, teen courts, tobacco courts, 
DUI courts, and family courts (Becker and 


Corrigan, 2002). Because these types of 


courts are so new, there is little research 
available on their effectiveness (MacKenzie 


2006: 224). An increasingly common type of 


These courts have also been referred to as 
“problem-oriented courts,’ “specialized courts,” 
“collaborative courts, and “therapeutic justice 
courts.” 


Stephen E. Vance 
Criminal Law Policy Staff 
Office of Probation and Pretrial Services 
Administrative Office of the U.S. Courts 


problem-solving court is the reentry court, 
which was first proposed as a method to man- 
age post-prison reintegration in the states in 
1999 by then-director of the National Institute 
of Justice Jeremy Travis. That year, Travis 
and then-Attorney General Janet Reno, who 
championed the country’s first drug court in 
1987 as a prosecutor in Dade County, Florida, 
announced federal support for state and local 
jurisdictions interested in establishing pilot 
reentry courts (Travis, 2005: vii). As Attorney 
General Reno explained, reentry courts apply 
drug court principles to the back end of the 
criminal justice system: 

[The reentry court] would oversee 

an offender's return to the commu- 

nity after release from prison or jail. 

The court will use its authority for 

positive reinforcement as drug courts 

do...This reentry court is modeled 

on the same theory of a carrot and 

stick approach [as drug courts] in 

using the strength of the court and 

the wisdom of the court to really 

push the issue...The reentry court 

would promote positive behavior 

by the returning offender. It would 

marshal resources to support the 

offender's successful reintegration 

into society. The court would also use 

its powers of punishment, using the 

graduated range of swift, predictable 

sanctions, to make sure that the indi- 

vidual stays on the right track. Judges 

working closely with others would 

approach or could approach a plan 

for reintegrating the offender into 
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the community. The court would 
then monitor and enforce the plan. 
The partners of court would include 
institutional and community correc- 
tional officers, law enforcement, local 
businesses, family, clergy, support 
services, victim advocates and neigh- 
borhood organizations (See Attorney 
General Janet Reno, Remarks at John 
Jay College of Criminal Justice on 
the Reentry Court Initiative (Feb. 
10, 2000), available at http://www. 
usdoj.gov/archive/ag/speeches/2000/ 
doc2.htm). 

In 2000, the Department of Justice’s Office 
of Justice Programs (OJP) launched the 
Reentry Court Initiative (RCI). Drawing on 
the drug court model, the goal of the RCI was 
to “establish a seamless system of offender 
accountability and support services through- 
out the reentry process” (Lindquist, et al., 
2004). The RCI solicitation identified six core 
elements of reentry courts—assessment and 
planning; active oversight; management of 
support services; accountability to commu- 
nity; graduated and parsimonious sanctions; 
and rewards for success. The OJP selected 
and provided technical support to nine states 
to implement pilot reentry courts. A process 
evaluation of the RCI concluded that, despite 
extensive variability across the nine sites, 
several lessons were learned. One of the most 
important conclusions was that it is essential 
to agree on the target population because, 
unlike drug courts (which involve focused 
treatment for offenders who share a common 
treatment need), “reentry courts that target 
the general population of returning offenders 
have to meet a diverse set of needs extend- 
ing far beyond substance abuse treatment” 
(Lindquist, et al., 2004). The evaluation con- 
cluded that further research is needed on the 
implementation, costs, and benefits of state 
reentry courts and other alternatives: 

Given that many of the [reentry 
court] programs are operating on 
a very small scale, it is particularly 
important to document the rela- 
tive costs and benefits of programs 
that serve a small number of par- 
ticipants. Additional research on the 
formation and functioning of alter- 
native models (including several 


* The OJP developed other reentry initiatives as 
well, such as the Reentry Partnership Initiative 
(RPI), which includes formation of a partnership 


between criminal justice, social service, and com- 
munity groups to develop and implement a reentry 
process in the states. 


non-court-based programs) and 
practices that are most effective with 
different types of offenders, would 
also be of importance. Given that 
reentry courts are a promising, yet 
fledgling approach to managing the 
complex problem of prisoner reen- 
try, documenting all types of models 
used in establishing programs is an 
important contribution to the field, 
one that will serve policymakers, 
practitioners, and researchers alike 
(Lindquist, et al., 2004). 


Reentry Court Programs 
in the Federal System 


A number of districts within the federal court 
system have established reentry court pro- 
grams over the past six years. Such programs 
allow the court to impose graduated sanctions 
and positive reinforcements in a team set- 
ting that typically involves a judge, probation 
officer, assistant United States attorney, assis- 
tant federal defender, and contract services 
provider. Within this general model, there 
is considerable variation. For example, some 
programs include only offenders with sub- 
stance abuse issues. Some courts accept only 
volunteers, whereas others mandate partici- 
pation by all offenders whom the officer and 
judge believe need intensive supervision. Some 
focus on offenders with a high probability of 
recidivism, as measured by the Risk Prediction 
Index score. Some involve informal monthly 
meetings with a judge, whereas others include 
formal status hearings in a courtroom (For an 
overview of the different types of federal reen- 
try court programs, see Meierhoefer, 2011, in 
this issue). As with state reentry courts, there 
is limited research on whether these programs 
effectively reduce recidivism. The following 
sections summarize evaluations for reentry 
court programs in the District of Oregon, the 
District of Massachusetts, and the Western 
District of Michigan. 


District of Oregon Reentry 
Court 


Program Description 

In 2005, the District of Oregon established 
one of the country’s first federal reentry 
court programs. The program is described 
and evaluated in a report titled The District 
of Oregon Reentry Court: Evaluation, Policy 
Recommendations, and Replication Strategies 
(Close, Aubin, and Alltucker, 2008). The study 
was written by researchers from the University 


of Oregon College of Education and from the 
court. As Close and his colleagues explain, 
the reentry court was created to address 
a public safety and health crisis caused by 
unprecedented levels of methamphetamine 
use. Oregon social services agencies were 
treating more individuals for methamphet- 
amine abuse per capita than any other state, 
and the District of Oregon's revocation rate 
rose above the national average. 

In response, the District of Oregon “initi- 
ated an aggressive campaign of research and 
study to address the challenges of drug abuse 
among the offenders under its supervision.” An 
Offender Treatment Committée* was formed 
to “gather information about best practices 
to address the crisis in drug abuse among the 
offenders in supervision.” In addition, the 
committee “sought information on innovative 
treatment programming and the procurement 
of increased funding for a range of treatment 
efforts.” Finally, it “sought to foster meaningful 
partnerships with a range of treatment agen- 
cies, the Federal Bureau of Prisons, and state 
and county social service agencies.” 

The reentry court program was designed 
based on six foundational principles: 

1. Transitional planning; 

2. Multidisciplinary training in evidence- 
based practices for the reentry court judge; 

3. The use of an integrated case management 
and law enforcement perspective for the 
reentry court probation officer; 

4. The research-informed use of monitoring, 
sanctions, and rewards; 

5. The research-informed use of a con- 
tinuum of services designed to enhance 
accountability and reduce barriers to 
reentry; and 

6. The establishment of quality data collec- 
tion and evaluation systems to measure the 
effectiveness of the reentry court program 
at the individual and community levels. 


The reentry court team comprises a fed- 
eral district judge, a probation officer, an 
assistant U.S. attorney, an assistant federal 
public defender, a drug and alcohol treat- 
ment professional, and a community services 
coordinator.’ It operates in a non-adversar- 
ial manner, encouraging, challenging, or 


3 


The Committee included representatives from 
the District of Oregon Court, the Probation Office, 
Federal Public Defenders Office, and the U.S. 
Attorney's Office. 

As Close and his colleagues explain, this descrip- 
tion is based on the reentry court model in Eugene. 
There may be variations between the programs in 
Eugene and Portland. 


re 


=f 


66 FEDERAL PROBATION 


Volume 75 Number 2 


sanctioning the participant in ways that depart 
from their conventional roles. The program 
includes court-mandated monitoring and 
community supervision, coupled with “indi- 
vidualized and effective long-term treatment 
and independent living supports coordinated 
by the probation officer.” This alternative to 
traditional release and supervision “provides 
the reentry court participant with the special- 
ized expertise of the courts and treatment 
services designed to encourage personal satis- 
faction and successful reintegration.” 

Participants voluntarily enter the reentry 
court after waiving certain due process rights. 
The program requires participants to enter 
into, and abide by, the terms of a contract. 
The participant acknowledges a willingness to 
comply with the terms of the individualized 
reentry plan and submit to periodic, random 
urinalysis and other monitoring. Sanctions 
are immediate and proportional to the offense 
and are designed to teach accountability and 
encourage progress in the participant's reentry 
plan. This procedure departs significantly 
from a traditional violation hearing. Where 
the participant's offenses fall short of the 
severity required to terminate him or her from 
the reentry court, the sanctioning process 
“encourages the participant to reflect on his or 
her mistake and correct it, without irreversibly 
interrupting progress toward the eventual goal 
of reentry success.” If a participant's offense 
warrants termination from the reentry court, 
he or she is transferred to conventional viola- 
tion proceedings, where the sanctions may be 
more serious. 

The reentry court team reviews each par- 
ticipant’s progress on a monthly basis under 
the direction of the judge and the leadership 
of the probation officer. Before monthly hear- 
ings, the probation officer prepares a detailed 
report on each participant, which updates the 
reentry court team on the individual's progress 
in substance abuse treatment, mental health 
therapy and counseling services, vocational 
training and job placement assistance, hous- 
ing assistance, education and training, and 
family counseling. Monthly hearings focus 
on “identifying the participant's needs and 
engaging the participant in problem-solving 
activities designed to meet those needs while 
preserving public safety and accountability.’ 
The needs assessment process is continuous 
and constantly changes over the course of a 
participant's involvement in the program. 

The reentry court hearing is an interactive 
set of discussions, often led by the judge, pro- 
bation officer, and participants. Typically, each 


participant undertakes a self-assessment after 
hearing the probation officer’s report, and the 
team engages the participant in problem-solv- 
ing strategies that target individual barriers to 
reentry and strategies for long-term success. 
Graduates of the program also participate 
in reentry court hearings, encouraging par- 
ticipants and assisting in the development of 
useful approaches to sobriety and desistance. 
The “cooperative nature of the reentry court 
approach provides an opportunity for the par- 
ticipant to change behaviors that led to his or 
her incarceration and to chart a new life that 
is clean, sober, and fully integrated into the 
life of the community.” Participants success- 
fully complete the program upon achieving 12 
continuous months of sobriety, as evidenced 
by random urinalysis testing. Graduates are 
honored with a ceremony involving partici- 
pants, other graduates, and supportive family 
and friends, and are eligible for a one-year 
reduction in the term of supervision. 


Evaluation Overview 


The District of Oregon Reentry Court “ini- 
tiated a comprehensive evaluation of its 
program and services during the early plan- 
ning phase of program development activities” 
to provide “both quantitative and qualitative 
information [on its effectiveness] from two 
perspectives: the participant's success in the 
community and the protection of public 
safety in the community” (Close, Aubin, and 
Alltucker, 2008). A total of 114 people were 
included in the study. There were 28 people 
in a “Comparison Group” (comprising indi- 
viduals under traditional supervision outside 
the reentry court context), 25 people in the 
“Current Reentry Court Participants Group,’ 
31 people in the “Reentry Court Graduates 
Group,’ and 30 people in the “Reentry Court 
Terminators Group.” 

Probation officers from Portland and 
Eugene selected the comparison group partic- 
ipants and identified the current, graduated, 
and terminated reentry court participants. 
Data were collected on eight different domains 
of information contained in the court files: 
demographics, sentencing, supervision, 
family, education, presentence information, 
criminal convictions, and chronological list 
(reentry court activities). Differences between 
the comparison, current reentry court par- 
ticipants, reentry court graduates, and reentry 
court terminators were calculated on four 
outcome variables: (1) total sanctions; (2) 
number of urinalyses; (3) number of positive 
urinalyses; and (4) the total number of support 


services used. Main effects were examined 
using one-way ANOVA, and post hoc analy- 
ses were performed with a Tukey test. The 
study calculated differences in employment 
status between the comparison group, cur- 
rent reentry court participants, reentry court 
graduates, and reentry court terminators 
using chi-square analyses. According to the 
study, “[s]ignificant differences were found 
among the Comparison, Current Reentry 
Court Participants, Reentry Court Graduates 
and Reentry Court Terminators on three 
outcome variables: total sanctions, number 
of urinalyses, and the number of support 
services used.” Specifically, the comparison 
group had the lowest average of total sanc- 
tions (.25) compared with the other groups. 
Current reentry court participants experi- 
enced an average of .92 sanctions, graduators 
had 1.6 sanctions, and terminators had the 
highest number of sanctions. 

There were also statistically significant dif- 
ferences found among groups on the total 
number of urinalyses performed. The compari- 
son group had the fewest number of urinalyses, 
with an average of 6.9. Current reentry court 
participants had an average of 21.7 urinalyses. 
The graduated group had the highest number 
of urinalyses with 22.1, and the terminators 
had an average of 18.6 urinalyses. The groups 
differed on the number of support services 
utilized as well. The comparison group par- 
ticipated in support services at the lowest 
level, compared with the other three groups. 
Participants in the comparison group used an 
average of 1.1 services, compared with 2.0 for 
the current reentry court group, 2.0 for the 
graduate group, and 1.9 for the terminated 
group. Groups did not significantly differ in 
the average number of positive urinalyses. 
According to the study, significant differences 
in employment status (yes/no) were found. 
For example, people in the comparison group, 
the current reentry court participants, and the 
graduators were more likely to be employed 
compared to the terminators. 

The study concluded that “it appears that 
the comparison group outperformed the 
treatment groups on multiple, important 
dimensions. For example, the comparison 
group underwent less monitoring and super- 
vision and had fewer drug and mental health 
services and yet had more employment and 
fewer sanctions.” The study authors warned 
that the study “has several limitations that 
restrict interpretation and generalizability 
of findings” relating to the initial design of 
the project, the simplicity of the outcome 
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measures utilized in the evaluation, the rela- 
tively small size of the sample, the limited 
duration of program efforts, the charac- 
teristics and demographics of the sample 
population, the constant changes in treatment 
procedures, the limitations in the availability 
of community services, and the constant 
improvement of skill of the professionals 
implementing the reentry court program. The 
authors also noted: 
We do not know whether the result 
of the evaluation is due to sam- 
pling error or some other flaw in 
the selection and measurement of 
this group...we know very little 
about individuals who are under 
conventional (non-reentry court) 
supervision...The fact that they 
had less contact with the court, the 
judge and probation officers, had 
less scrutiny of their actions, and 
less opportunity to be accountable 
points out the flaws in the initial 
design of the study. These indi- 
viduals are not being monitored as 
frequently or as intensively as the 
treatment group participants. This 
lack of information about the com- 
parison group is contrasted with the 
detailed and immediate informa- 
tion available regarding the reentry 
court participants. 


The District of Massachusetts 
Court Assisted Recovery Effort 


Program Description 
In May of 2006, the District of Massachusetts 


established the Court Assisted Recovery Effort 
(C.A.R.E.). The program is described and 


evaluated in a report titled Evaluation of 


the Court Assisted Recovery Effort (C.A.R.E.) 
Program - United States District Court for 
the District of Massachusetts (Farrell and 
Wunderlich, 2009). The study was written by 
researchers from the Northeastern University 
School of Criminology and Criminal 
Justice. As the study explains, state courts 
throughout Massachusetts have increasingly 
utilized drug courts as a means to combat 
the Commonwealth’s growing drug abuse 
problems. Cocaine and heroin are two of the 
primary drugs of abuse in Massachusetts, and 
opiate-related deaths are on the rise. Facing 


a “growing population of drug-involved 
offenders, the District of Massachusetts began 
looking for new and innovative mechanisms 
to address the challenges of supervising 
addicted offenders.” 

According to Farrell and Wunderlich 
(2009), the C.A.R.E. program “uses a modi- 
fied drug court program to provide enhanced 
supervision of offenders while addressing 
the problems that accompany addiction.” 
Offenders who have a significant drug abuse 
history and are serving terms of supervised 
release or probation voluntarily enroll in 
the program, subject to the approval of the 
court with input from the C.A.R.E. team. 
The principal goal of C.A.R.E. is to “trans- 
form offenders into sober, employed and 
law-abiding citizens.” The program involves 
“closer supervision of an offender and higher 
expectations than regular supervision, but 
it also offers an offender greater assistance, 
opportunity and reward.” Each participant is 
“challenged to accept responsibility for the 
impact of his or her addiction on others, and 
is provided the tools necessary to achieve and 
maintain sobriety.’ 

The program lasts at least 52 weeks, con- 
sisting of three 12-week phases and one 
16-week phase. The four phases are “Early 
Recovery, “Understanding and Taking 
Responsibility,’ “Healthy Decision Making,’ 
and “Relapse Prevention Planning.” Offender 
supervision is most intensive in the first 
phase, requiring participants to attend weekly 
court sessions and to appear in the probation 
office three times per week for meetings and 
drug tests. During this phase, participants are 
also expected to attend substance abuse and 
mental health treatment as deemed necessary 
and begin a life skills, employment, or edu- 
cation program. The second phase requires 
weekly meetings with the probation office and 
biweekly court appearances. Participants con- 
tinue a life skills, employment, or education 
program and participate in substance abuse 
and/or mental health treatment. 

The third phase of the program requires 
biweekly court attendance and meetings with 
the probation office and less frequent atten- 
dance at substance abuse and/or mental health 
treatment. By the end of this phase, participants 
are expected to have secured employment. The 
fourth and final phase requires participants to 
appear in court and at the probation office 


once per month. Participants are required to 
maintain employment, attend treatment when 
necessary, and complete an approved, written 
relapse prevention plan prior to graduation. 
Throughout the course of the program, the 
probation officer's contact with the offenders 
is not limited to office visits. The type and 
level of contacts vary based on the needs and 
risk level of individual offenders. 

Progression from one phase to another 
allows for more lenient supervision, but it is 
a “privilege that must be earned.” Participants 
are required to adhere to the terms of the 
program and the terms of supervision, or face 
a sanction and be deprived of credit for a par- 
ticular week or weeks. Examples of sanctions 
include writing assignments, a day or night in 
custody, loss of credit for the week, and com- 
munity service. Participants can also receive 
rewards weekly through acknowledgment of 
a successful week in open court and through 
certificates for completion of each phase. 

The court and the probation office encour- 
age each participant weekly to work on 
treatment issues and other goals. During the 
court session, each participant comes forward 
to engage in discussion with the judge about 
his or her progress or troubles. If there are any 
program or supervision violations, the judge 
imposes the sanction to be completed by the 
next court session, and the participant does 
not earn a good week. A “good week” means 
that the participant attended all required 
meetings with probation and all treatment 
sessions, submitted to all scheduled drug tests 
(and tested negative), and complied with all 
other conditions of supervision. Participants 
must earn 12 good weeks and complete a writ- 
ing assignment in order to progress through 
each of the first three phases. They must then 
earn 16 good weeks during the last phase and 
complete a relapse prevention plan in order 
to graduate the program. For more severe 
program violations, the offender may be ter- 
minated from the program or demoted to 
an earlier stage of the program. Allegations 
of new criminal conduct are brought to the 
district judge and are not addressed in the 
C.A.R.E. program. Upon successful comple- 
tion of the program, participants may receive 
up to a one-year reduction in the term of their 
supervised release. 

Participants volunteer for the program 
and are then screened and accepted by the 
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probation office. They generally score in the 
severe range on the Risk Prediction Index 
(RPI)° and Texas Christian University (TCU) 
Drug Screen.° Additionally, C.A.R.E. par- 
ticipants are required to demonstrate a serious 
history of substance abuse, typically do not 
have mental health issues as their primary 
diagnosis, and cannot be not registered as 
a sex offender. Individuals are admitted to 
C.A.R.E. on a rolling basis. The court (a mag- 
istrate judge), the probation office, the United 
States attorney's office, the federal defender’s 
office, and outside treatment contractors all 
participate in the program in a joint effort 
to help each offender accomplish his or her 
goals. 

Before each weekly court session, the mag- 
istrate judge meets with this team to review 
the status of each participant in the program 
and to discuss any changes in treatment, 
compliance problems, or suggested sanctions. 
The representatives from each office “work 


Participants of the C.A.R.E. program were admin- 
istered the Risk Prediction Index (RPI) developed 
by the Federal Judicial Center for assessing federal 
offenders with regard to their risk of recidivism. 
Along with the start date of supervision, items on 
the RPI include: (1) the offender’s date of birth, 
(2) the number of prior arrests before the instant 
offense (up to 15 arrests), (3) the use of a weapon 
in the instant offense, (4) whether the offender is 


employed at the start of supervision, (5) history of 


illegal drug or alcohol abuse, (6) previous abscond- 
ing while on supervision, (7) whether the offender 
has graduated from college, and (8) whether the 
offender is residing with a spouse or children at the 
start of supervision. Offenders can score a maxi- 
mum of 9 on the RPI. The three risk categories on 
the RPI correspond to the following scores: (1) Low 
risk scores range from 0-2, (2) Moderate risk scores 
range from 3-5 and (3) High risk scores range 
from 6-9. Offenders eligible for participation in the 
C.A.R.E. program are required to score between 6 
and 9 on the RPI. 


® The original TCU Drug Screen was developed by 
researchers at the Institute of Behavioral Research 
at Texas Christian University. It includes 19 items 
that represent key clinical and diagnostic criteria 
for substance “dependence” as they appear in the 
Diagnostic and Statistical Manual and the National 
Institute of Mental Health Diagnostic Interview 
Schedule. The first part of the TCU Drug Screen 
includes a series of 10 questions about problems 
related to “drug use,’ and the second part addresses 
the frequency of specific drug use prior to prison 
as well as a self-assessment of one’s readiness for 
substance abuse treatment. Based on the first nine 
items, a continuous composite score is computed 
that measures the level of an offender’s drug use 
severity. The remaining TCU Drug Screen items 
are designed to provide corroborative evidence 
of potential drug use problems, such as questions 
pertaining to prior drug treatment. 


together to provide encouragement when 
participants are excelling, and to respond 
effectively when participants are in non-com- 
pliance by providing immediate intervention 
and promptly addressing the issues.” The 
active involvement of each team member 
is “essential to not only the success of the 
program, but also to the success of each indi- 
vidual participant.” Program participants are 
also expected to “provide support to their 
peers in the program and receive the benefits 
of an extended network of group accountabil- 
ity and encouragement.” 


Evaluation Overview 


Northeastern University’s Department 
of Criminology and Criminal Justice con- 
ducted an evaluation of C.A.R.E. (Farrell 
and Wunderlich, 2009). The study examined 
whether participants in the C.A.R.E. program 
were more successful than a comparison 
group of offenders who receive traditional 
supervision across three main measures: (1) 
their ability to remain sober, (2) their being 
employed, and (3) their being law abiding. 
The study utilized data collected through U.S. 
Probation resources about C.A.R.E. partici- 
pants and comparison group progress across 
a number of outcome measures collected 
at intervals of three, six, and twelve months 
between May 2006 and May 2009. 

C.A.R.E. participants were not randomly 
assigned to the program but volunteered. 
Therefore, efforts were made to help ensure 
the equivalence of the treatment and com- 
parison groups. To select comparison group 
members, in May 2006 the probation office 
compiled a list of eligible federal offenders in 
Massachusetts based on their RPI and TCU 
scores and drug abuse history. Offenders 
selected for assignment into the comparison 
group could not have serious mental health 
problems and could not be registered as sex 
offenders. A list of eligible participants who 
were not actively participating in C.A.R.E. was 
compiled and the comparison group was ran- 
domly selected from this list. This process was 
replicated in 2007 and 2008. Offenders who 
were part of the participant or comparison 
groups in the previous years were excluded 
from the subsequent comparison group sam- 
pling processes. 

As to the data collection methods, at 
months three, six, and twelve of an offender's 
participation in the program, the proba- 
tion office collected data on the status and 
progress of both C.A.R.E. and comparison 
group members concerning new charges, 


revocations, compliance issues, employment 
status, number of positive and negative urine 
analysis tests, current drug and alcohol usage, 
mental health issues, family and residential 
status, and services or treatment received. 
The study’s author also obtained Criminal 
Offender Record Information (CORI) from 
the probation office for each individual in the 
treatment and comparison population. The 
CORI records were utilized to determine if 
a participant had received any new charges 
through June of 2009. This information was 
used to calculate the number and type of new 
charges for each participant. Information 
from presentence investigation reports was 
also obtained to provide more detail about 
the prior criminal history, marital status, 
number of dependents, education and prior 
employment, and family circumstances of 
study participants. 

In total, 46 offenders participated in 
C.A.R.E. between May 2006 and May 2009. 
Sixty-eight comparison group members were 
selected for inclusion in the study during 
this period. Descriptive statistical analyses 
“contirmed the similari- 
ties between the treatment and comparison 
groups across characteristics of interest to the 
research.” Chi-Square tests and t-tests were 
used to compare the demographic and offense 
history characteristics of C.A.R.E. and com- 
parison group participants. The study found 
“only a few statistically significant differences 


of the two samples 


between the demographic and offense history 
characteristics of treatment and comparison 
group members.” There were proportionately 
more black participants in the treatment 
group (32.6 percent) than in the comparison 
group (19.1 percent), though the difference 
was only marginally significant (p=.0.08). 
There were proportionately more Hispanic 
members of the comparison group (25.6 
percent) compared to the treatment group 
(8.7 percent). While the treatment and con- 
trol groups were similar in the average age of 
onset for drug use, 15.2 percent of treatment 
group participants began using drugs before 
the age of 12 compared to only 4.3 percent 
of comparison group participants. The risk 
assessment scores, TCU scores, and previous 
criminal histories were similar for treatment 
and comparison group members. RPI scores 
were nearly identical for the two groups (5.9 
for CARE and 5.8 for the comparison group). 
On average, TCU scores were slightly higher 
for the CARE participants (7.4) than for the 
comparison group members (6.5). Treatment 
group members also had more juvenile and 
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adult convictions than did comparison group 
members. Overall, the study “found similar 
patterns in the demographic characteristics 
and offense histories of treatment and com- 
parison group members in specific cohort 
years as [it] found overall” 

Turning to the outcome analysis, the study 
found that C.A.R.E. group participants were 
“more likely to meet the standards necessary 
to graduate (12 months of consecutive no new 
charges, employed, and no positive drug tests) 
than the comparison group.’ Approximately 
46 percent of the C.A.R.E. group members 
met the standards for graduation, compared 
to only 31 percent of the comparison group 
members. Because the study used a .10 level 
of statistical significance, it included the fol- 
lowing caveat: “While studies measuring 
differences between groups traditionally use 
.05 or .01 thresholds for significance, the small 
sample sizes allow some latitude to examine 
differences between groups that fall below the 
.10 level. While the differences are likely not 
due to chance alone, they are also not over- 
whelmingly strong.” 

The researchers also conducted multivari- 
ate analyses (logistic regression) on whether 
program participation increases the likeli- 
hood of a successful graduation outcome 
(12 consecutive months of no new charges, 
employed, and no positive drug tests), con- 
trolling for cohort year and risk prediction 
scores. They concluded that participation 
in C.A.R.E. “is significantly associated with 
successful outcomes.” When controlling only 
for the year of participation, participants 
in C.A.R.E were statistically significantly 
more likely to have successful outcomes than 
comparison group members (p=.02), and 
the odds of success for C.A.R.E. participants 
were 2.6 times greater than for comparison 
group members. When the researchers also 
controlled for offender characteristics in the 
regression model, they found that the odds of 
success for C.A.R.E. participants decreased 
slightly (from 2.60 to 2.28), but none of the 
personal characteristics of offenders signifi- 
cantly predicted successful graduation. The 
authors warned, however, that the regression 
models should be interpreted cautiously due 
to the small number of cases. 

When the authors separately examined 
each component of successful graduation (no 
new charges, employed, and no positive drug 
tests) they found more similarities than dif- 
ferences between C.A.R.E. and comparison 
group members. For instance, 6.8 percent of 
the C.A.R.E. group had a new charge, while 


10.8 percent of the comparison group had a 
new charge. With regard to employment, 43.2 
percent of the C.A.R.E. group was employed, 
while 47.1 percent of the comparison group 
was employed. As to positive drug tests, 51.1 
percent of the C.A.R.E. group had a positive 
drug test, while 40.3 percent of the compari- 
son group tested positive. The authors stressed 
that “[it] is important to note that [this data] 
includes new charges, employment and posi- 
tive drug tests over the full 24-month period 
of data collection, and “[i]n some cases, 
C.A.R.E. and comparison group members had 
a failure on one of the measures in the first 
year but were able to complete a second year 
of supervision successfully.” 

Because C.A.R.E. participants had the 
opportunity to continue in the program and 
repeat program phases if they encountered 
early setbacks, the researchers developed a 
second outcome measure that examines any 
negative outcomes in the last 12 months of 
data collection for each group. They found 
that C.A.R.E. participants were “significantly 
more likely to be employed than comparison 
group members [were] in the final 12 months 
of data collection. The difference between 
groups in receiving new charges or having 
positive drug tests [was] non-significant in the 
final 12 months of data collection” 

While C.A.R.E. participants were more 
likely to have positive urine analysis results 
than comparison group members in both the 
overall and last 12 months measures (though 
the differences were not statistically signifi- 
cant), the researchers stressed that the groups 
were not equivalent in terms of the risks for 
failure. The participants underwent far more 
frequent drug screening (tested 35 times on 
average) than comparison group members 
(tested 9 times on average). To account for the 
different risks for failure, the study authors 
computed the average proportion of positive 
urine analysis tests to total tests and found that 
participants were less likely than the compari- 
son group to test positive in proportion to the 
total number of tests given. 

In their concluding section, the research- 
ers state that C.A.R.E. participants “have 
greater likelihood of overall success—mea- 
sured here as law abiding, employed, and 
sober—than participants in a comparison 


group who received only normal supervision.” 
They noted, however, that participants faced 
“struggles throughout the study period.” For 
instance, some cohorts of C.A.R.E. partici- 
pants were more likely than the comparison 
group to have positive drug tests. The authors 


attribute this to the fact that participants, on 
average, underwent nearly four times as many 
drug tests. Despite these challenges, C.A.R.E. 
participants were more likely to have the full 
combination of positive outcomes, which is 
a “sign that participation in C.A.R.E. assists 
offenders holistically.” 

The authors also discussed two “important 
limitations.” First, the number of participants 


in the treatment and control groups was 


small. As a result, the study findings are “not 


particularly strong,’ and “a few cases in one 


direction or another might change outcomes 
of our analysis, for example rendering a statis- 
tically significant result to be non-significant.” 
Second, while the analyses indicated that 


participants were “at least marginally more 
successful at avoiding new charges, securing 


employment and remaining drug-free than 
a comparable group of offenders under tra- 
ditional supervision, we do not know why.’ 


The researchers suggest a number of potential 
explanations for these findings, guided by 
theory and prior research. For instance, the 
blending of “a public health and criminal 
justice approach may improve both treatment 


and accountability.” It is also possible that par- 


ticipation in C.A.R.E. fosters “social support 
networks that aid offenders as they transition 
through supervision.” Finally, the “judicial 


oversight and encouragement of C.A.R.E. 
may also more effectively keep program par- 
ticipants in treatment than regular supervision 


because of its early intervention model.” The 
study concluded: “Further understanding 
why C.AR.E. participants are successful, and 
identifying areas where they struggle will 


ultimately help court officials improve and 


expand the program to better address the 


needs of addicted offenders.” 


Western District of Michigan 


Program Description 


In 2005, the Western District of Michigan 
established the Accelerated Community Entry 
Program (ACE). The program is described 
and evaluated in a report titled An Evaluation 
of the Accelerated Community Entry Court 
Program (Lowenkamp and Bechtel, 2010). 
The study was written by researchers from 
the Administrative Office of the U.S. Courts 
and the University of Cincinnati’s Center for 
Criminal Justice Research. The program was 
initially established in Benton Harbor but 
has since expanded to two additional sites 
in Kalamazoo and Grand Rapids. It targets 
high-risk offenders following release from 
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prison and uses a multi-disciplinary approach 
to address the needs of the participants (Evers 
and Martin, 2007). 

The collaborative partners include the 
district court, the probation office, the U.S. 
attorney's office, the federal defender’s office, 
the Federal Bureau of Prisons, local commu- 
nity service providers, and the participants’ 
prosocial support system (family, friends, 
significant others, employers, and clergy). 
The ACE program model “calls for the use 
of evidence-based practices,’ including the 
use of assessment practices, to identify and 
target high-risk offenders and their specific 
criminogenic needs, addressing the offender's 
motivation for behavioral change, providing 
prosocial models and supports to encourage 
law-abiding behaviors, monthly court super- 
vised status hearings, participation in Moral 
Reconation Therapy (MRT), and the use of 
rewards and sanctions as appropriate (Evers 
and Martin, 2007). 

Federal probation officers identify pro- 
gram candidates based on their Risk 
Prediction Index score. Offenders eligible 
for participation must score between 6 and 9 
on the RPI.’ Potential participants must also 
complete a contract that identifies the condi- 
tions of supervision, the system for rewards 
and sanctions, and the criteria for successful 
completion of the program. Participants are 
not unsuccessfully terminated from the ACE 
program. Should revocation occur on supervi- 
sion, the individual starts over in the program 
after incarceration. Offenders who choose 
not to participate are required to seek judicial 
approval to be removed from the program. 

A standard requirement of ACE is atten- 
dance at the monthly court status hearings, 
where the ACE team members describe the 
offender’s progress. A report summarizing 
the offender's monthly progress, goals for 
the following month, and rewards or sanc- 
tions received is completed at the end of each 
status hearing. Violations of supervision are 
addressed either at the time of the ACE court 
hearing or before the hearing, depending 
on the severity of the violation. Successful 
discharge from the program occurs when the 
participant accrues 12 months of rewards. 
Along with receiving a certificate of comple- 
tion, the graduation process involves placing 
the offender on traditional supervision status 
for an additional 12 months. Following this 


A small number of moderate- and lower-risk 
offenders entered ACE at the onset of the program. 
This may indicate some drift in the eligibility 
requirements for entry into the ACE program. 


time period, the officer can request early 
termination of supervision for good behavior. 


Evaluation Overview 


Researchers from the Administrative Office 
of the U.S. Courts and the University of 
Cincinnati’s Center for Criminal Justice 
Research conducted an evaluation of the ACE 
program (Lowenkamp and Bechtel, 2010). The 
purpose was to “provide some initial outcome 
results” related to the program participants. 
The sample size for the preliminary analysis 
consisted of 36 ACE participants. While there 
had been 77 participants at the time of the 
evaluation, many of them were removed from 
the analysis because they started the program 
too late for a full follow-up period. (A more 
comprehensive evaluation will be completed 
in the future as more offenders complete the 
program and more time has passed to allow 
an analysis of an appropriately long follow- 
up period.) Data for the sample were pulled 
from the Probation and Pretrial Services 
Automated Case Tracking System (PACTS). 
The primary variables included general demo- 
graphics (sex, race, ethnicity, and age), RPI 
total score and risk category, motivation, 
supervising officer, supervision location, par- 
ticipation in Moral Reconation Therapy, and 
re-arrest.* The follow-up time period for re- 
arrest was 12 months. The re-arrest measure 
was defined as (1) re-arrest for a new crime 
and/or technical violation and (2) re-arrest for 
a new crime. 

For the study samples, the researchers used 
a comparison group of 121 offenders that did 
not participate in the ACE reentry court pro- 
gram. Each of the offenders in the comparison 
group was supervised by one of the same four 
probation officers that were responsible for 
supervising ACE participants. Several charac- 
teristics of the treatment group were used to 
select the comparison group. Three of these 
variables were constants: being male, non- 
Hispanic, and not a sex offender. The fourth 
variable, race, was defined as being either 
Black or White. Of greater importance, the 
researchers also conducted a matched sample 
analysis (36 ACE participants and 36 compari- 
son group members), where offenders were 
matched, not only on demographic factors, 


* The motivation measure is an item that comes 


from PACTS that asks the officer to rate the 
offender's motivation toward making changes and/ 
or participate in supervision. MRT data were taken 
from the Administrative Office of the U.S. Court’s 
Decision Support System and re-arrest data were 
taken from the judicial re-arrest file compiled by 
ABT Associates. 


but also on risk (using the RPI), motivation, 
and supervising officer.’ 

Statistical analyses were in three phases: 
(1) matched samples; (2) unmatched samples; 
and (3) treatment group only. The analysis of 
the matched samples (N=72) was limited to 
the calculation of frequencies to describe the 
sample, t-tests, and chi-square tests to exam- 
ine the differences between the treatment and 
comparison groups and bivariate analysis to 
examine the outcome measure, re-arrest.'° For 
the unmatched sample (N=157), the statistical 
analysis was similar to the matched sample, 
with the addition of a multivariate logistic 
regression model controlling for motivation, 
risk, age, and group membership. Predicted 
probabilities were also calculated from the 
multivariate model. Finally, an additional 
bivariate analysis conducted on treatment 
group members only examined participation 
in Moral Reconation Therapy (MRT) and 
re-arrest. 

For the matched sample analyses, 
descriptive statistics revealed no statistically 
significant differences between the ACE and 
non-ACE participants. The researchers con- 
ducted a bivariate analysis examining the 
differences between the treatment and com- 
parison groups with regard to re-arrest (new 
crime and technical violation) at 12 months. 
Nearly 40 percent of the treatment group 
(ACE participants) was re-arrested at the 
12-month follow-up period in comparison to 
58 percent of the matched cases that did not 
participate in the ACE program. Chi-square 
analysis revealed that the difference between 
the groups with regard to re-arrest was sta- 
tistically significant (p=.099). The study also 
included a bivariate analysis comparing new 
crime re-arrest between group membership. 
One quarter of the treatment group was re- 
arrested for a new crime compared to one half 
of the non-ACE participants, a finding that 
was also statistically significant (p=.028). 

Turning to the unmatched sample analysis 
of the larger comparison sample (N=121), 
descriptive statistics revealed that there were 
statistically significant differences between 
the unmatched samples for age, race, and 


° While there were some differences with exact 


matching for race, supervising officer, and moti- 
vation for the matched sample analysis, the risk 
category from the RPI was considered to be the 
most important factor to hold constant through the 
matching of the groups. 


'° For significance testing, the p-value was set at .10 
due to the relatively small sample size. 
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risk level.'' As with the matched sample 
analysis, a bivariate analysis was performed 
comparing the 12-month re-arrest rates for 
the unmatched sample. The percentage of 
comparison group members that were re- 
arrested at the 12-month follow-up period 
was 49 percent and the re-arrest rate (includ- 
ing technical violations) for the treatment 
group was 39 percent, though chi-square test 
results indicated that the difference was not 
statistically significant. Similarly, when using 
arrest for new criminal arrests only (arrests for 
technical violations excluded), the differences 
were not statistically significant. 

The unmatched sample analysis also 
included a multivariate logistic regression 
model predicting re-arrest (including tech- 
nical violations) while controlling for age, 
motivation, RPI moderate- and high-risk 
categories, and group membership. Based 
on these results, members of the compari- 
son group were significantly more likely to 
be re-arrested at the 12-month follow-up 
period. Motivation was found to be a signifi- 
cant predictor of re-arrest, with unmotivated 
offenders being significantly more likely to 
experience re-arrest. Collectively, the RPI 
categories were found to be a significant pre- 
dictor of re-arrest (although the individual 
risk categories were not), with increases from 
one risk category to the next representing an 
increase in the likelihood that an offender 
will be re-arrested. An offender’s age was 
also shown to be significantly related to re- 
arrest, with younger offenders significantly 
more likely to be re-arrested. The multivariate 
model predicting arrests for criminal behavior 
only (excluding technical violations) indicated 
that treatment group and age were the only 


'' The average age of the comparison group was 


nearly 36 years of age. The average RPI score was 
nearly 6 for the comparison group. T-tests examin- 
ing the differences between the unmatched samples 
suggested that there were significant differences 
between the treatment and comparison groups 
with respect to age and RPI score. Recall that the 
average RPI score for the treatment group was 
slightly over 7 and the average age of the treatment 
group members was nearly 32 years. As such, the 
comparison group members were lower risk in 
terms of age and total RPI score. Regarding race, 
86 percent of the comparison group is identified as 
Black and 14 percent are White. According to the 
RPI risk categories, 15 percent of the comparison 
group is low risk, 24 percent are moderate risk and 
61 percent are high risk. For motivation, 38 percent 
of the comparison group was identified as being 
motivated toward supervision and 62 percent were 
unmotivated. Based on chi-square test results, there 
is a significant difference between groups in this 
unmatched sample for race and RPI risk category. 


statistically significant predictors, with the 
effect of treatment (ACE participation) being 
the same magnitude as in the multivariate 
model predicting re-arrest (including techni- 
cal violations).!” 

The final analysis focused on ACE par- 
ticipants only. It compared the 12-month 
re-arrest rates of ACE participants that par- 
ticipated in Moral Reconation Therapy (MRT) 
and those that did not participate. The results 
of this bivariate analysis suggested that 50 
percent of the MRT participants were rear- 
rested, compared to a 21 percent re-arrest rate 
for the group that did not participate in MRT. 
Chi-square analysis findings indicated that 
the difference between participants and non- 
participants for 12-month re-arrest rates was 
statistically significant. However, the study 
noted that this finding should be interpreted 
cautiously due to the small sample size. 

The concluding section of the study 
highlighted several key findings. First, the 
participants of the ACE program “appeared 
to have lower recidivism rates as measured by 
re-arrest in a 12 month follow-up period than 
the offenders that did not participate in ACE.” 
The study stated that, “[w]hile encouraging, 
these findings are considered preliminary 
due to the small sample size and one year 
follow-up period.” Second, motivation toward 
supervision “repeatedly was demonstrated 
as a significant variable with regard to lower 
recidivism rates” in “both the matched and 
unmatched sample analysis.” This finding 
“provides additional support to the research 
on motivation and the importance of assessing 
for motivation and providing pre-treatment 
for offenders who are identified as not moti- 
vated for participation in programming.” 

Third, risk, as measured by the Risk 
Prediction Index, was shown to be a “sig- 
nificant predictor of re-arrest indicating that 
the higher risk offenders were more likely 
to recidivate.’ Given that the RPI categories 


Predicted probabilities were also calculated 
for the significant predictors of re-arrest from the 
multivariate model to provide specific examples of 
how ACE participation might affect re-arrest rates. 
Unmotivated, high-risk, 33-year-old offenders that 
did not participate in the ACE program would 
potentially experience a 73 percent re-arrest rate. 
In contrast, the re-arrest rate would be 44 percent 
for unmotivated, high-risk, 33-year-old offenders 
that did participate in ACE. A similar trend was 
noted for motivated offenders. More specifically, 
motivated, high-risk, 33-year-old offenders that 
did not participate in ACE would experience a 47 
percent re-arrest rate in comparison to 21 percent 
for the motivated, high-risk, 33-year-old ACE par- 
ticipants. 


were a significant predictor for re-arrest and 

in light of prior research demonstrating that 

“the mixing of risk levels has been shown to 

increase the recidivism rates for the lower 

risk offenders,’ the study recommended “con- 
tinued efforts to avoid mixing risk levels.” 

Fourth, youthful offenders were more likely 

to experience re-arrest. Finally, participation 

in MRT yielded higher re-arrest rates for ACE 
offenders than for those that did not partici- 
pate in MRT." 

Finally, the researchers discussed several 
of the preliminary study’s limitations. For 
instance, the sample sizes were “rather small 
and this serves as a limitation for the statistical 
analysis as well as the reliability and gener- 
alizability of the results.” Therefore, “caution 
should be observed regarding the reliabil- 
ity of significant findings and the ability to 
generalize these findings to other offending 
populations.” In addition, only males and 
non-Hispanics were included in this analysis, 
thereby limiting the generalizability to females 
and Hispanic offenders. Finally, multivariate 
models controlling for motivation, age, risk, 
and group membership could only be con- 
ducted on the unmatched sample due to not 
having a large enough matched sample. 

The authors of this preliminary study listed 

several recommendations for the next evalu- 
ation, which is expected to be completed in 
the future when additional data are available. 
These recommendations included: 
1. Continuing data collection to increase the 
sample size and representativeness; 
Conducting additional process and 
outcome analyses to examine what 
characteristics of ACE participants are 
statistically associated with successful 
completion of the program as well as 
reductions in recidivism when compared 
to a matched group that did not partici- 
pate in ACE; 

3. Examining the impact of ACE on re-arrest 
by risk level based on the RPI; 

4. Considering a plan to examine the effec- 
tiveness of MRT in reducing recidivism, 
which may include a study examining pro- 
gramming characteristics, such as capacity 


N 


The study warned, however, that it was unclear 
based on available data if treatment providers cor- 
rectly adhered to the MRT curriculum. Moreover, 
prior research has suggested that MRT is most suc- 
cessful in programs implemented by the creators or 
developers of the curriculum and that it is unclear 
whether criminal justice personnel can success- 
fully deliver the necessary MRT programming. 
Therefore, the study recommended further research 
into the issue of whether MRT can be delivered as 
intended. 
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and content, and might also examine MRT 
by individual vendors to see if there are 
different effects by providers, suggesting 
issues related to program fidelity; 

5. Identifying other recidivism measures to 
examine for the ACE population, which 
may include return to prison on a new 
conviction; 

6. Extending the follow-up period for mea- 
suring recidivism, since treatment effects 
generally diminish over time, and explor- 
ing options for behavioral maintenance 
(e.g., relapse prevention plans) once 
offenders have left the ACE program and 
supervision; and 

7. Identifying whether each ACE program 
site is operating with fidelity to the pro- 
gram design. 


Summary and Conclusion 


A number of districts within the federal 
court system have established reentry court 
programs over the past six years. These pro- 
grams allow the court to impose graduated 
sanctions and positive reinforcements in a 
non-adversarial team setting. Because reentry 
court programs are so new in both the state 
and federal systems, there is little empirical 
research on whether they successfully reduce 
recidivism. This paper has summarized stud- 
ies evaluating three of the earliest federal 
reentry court programs. 

In 2005, the District of Oregon estab- 
lished a reentry court program to address 
unprecedented levels of methamphetamine 
use and a high supervision revocation rate. 
The program was evaluated by researchers 
from the University of Oregon College of 
Education and from the court (Close, Aubin, 
and Alltucker, 2008). A total of 114 people 
were included in the study. The authors con- 
cluded that “it appears that the comparison 
group outperformed the treatment groups on 
multiple, important dimensions. For exam- 
ple, the comparison group underwent less 
monitoring and supervision and had fewer 
drug and mental health services and yet had 
more employment and fewer sanctions.” The 
authors warned that the study has several 
limitations that restrict interpretation and 
generalizability of findings, such as the small 
sample size ard initial design of the project. 

In 2006, the District of Massachusetts 
created the Court Assisted Recovery Effort 
(C.A.R.E.) to address a growing population 
of drug-involved offenders. The program was 
evaluated by researchers from the Northeastern 
University School of Criminology and 


Criminal Justice (Farrell and Wunderlich, 
2009). The study examined whether par- 
ticipants in the C.A.R.E. program were more 
successful than a comparison group of offend- 
ers who receive traditional supervision. The 
study included 46 offenders who participated 
in C.A.R.E. and 68 offenders in the com- 
parison group. The authors concluded that 
C.A.R.E. participants “have greater likelihood 
of overall success—measured here as law abid- 
ing, employed and sober—than participants 
in a comparison group who received only 
normal supervision.” The authors also dis- 
cussed important limitations, including the 
small number of participants in the treatment 
and control groups. As a result, the study 
findings were “not particularly strong,’ and “a 
few cases in one direction or another might 
change outcomes of [the] analysis, for example 
rendering a statistically significant result to be 
non-significant.” 

In 2005, the Western District of Michigan 
established the Accelerated Community 
Entry Program (ACE). Researchers from 
the Administrative Office of the U.S. Courts 
and the University of Cincinnati’s Center for 
Criminal Justice Research conducted an eval- 
uation of the ACE program to provide some 
preliminary outcome results (Lowenkamp 
and Bechtel, 2010). The sample size for the 
preliminary analysis consisted of 36 ACE 
participants. The comparison group consisted 
of 121 offenders (for the unmatched sample 
analysis) and 36 offenders (for the matched 
sample analysis). One major finding of the 
study was that the participants of the ACE 
program appeared to have lower recidivism 
rates, as measured by re-arrest in a 12-month 
follow-up period, than the offenders not par- 
ticipating in ACE. The authors also found that 
motivation toward supervision was repeat- 
ediy demonstrated as a significant variable 
for lower recidivism rates. Finally, risk, as 
measured by the Risk Prediction Index, was 
shown as a significant predictor of re-arrest, 
with higher-risk offenders being more likely 
to recidivate. The researchers stressed several 
study limitations, such as the sample size, 
which limits the statistical analysis, reliability, 
and generalizability of the results. 

The studies of federai reentry court pro- 
grams discussed above provide mixed results 
on whether the programs effectively reduce 
recidivism. The evaluators of the District of 
Oregon Reentry Court found that the com- 
parison group outperformed the treatment 
group on multiple important dimensions. 
However, as the study notes, this conclusion 


should be interpreted with caution due to 
factors such as the small sample size and 
the initial project design. The studies of the 
District of Massachusetts’ C.A.R.E. program 
and the Western District of Michigan's ACE 
program found that the reentry court program 
participants were more likely to have positive 
outcomes, though the authors also stressed 
that these findings should be interpreted cau- 
tiously due to such limitations as the small 
sample size. 

As discussed above, when Lindquist and 
her colleagues (2004) were commissioned by 
the National Institute of Justice to conduct 
a process evaluation of the Reentry Court 
Initiative of the Office of Justice Programs, 
they stressed that, given the small number of 
participants in state reentry court programs 
and the fact that these programs were so new, 
it is important to document the relative costs 
and benefits of both the court-based and 
non-court-based programs for managing the 
complex problem of prisoner reentry. This 
recommendation should apply to the federal 
criminal justice system as well. 

In the future, studies should continue to 
examine whether reentry court programs and 
other non-court-based practices effectively 
reduce recidivism. It is particularly important 
that future research use larger sample sizes 
to improve the validity and reliability of the 
findings. Whether evaluating a reentry court 
program or any other type of intervention, 
researchers might also consider examining 
the effect on recidivism based on offender 
risk level. A large body of research has shown 
that criminal justice interventions are more 
effective on higher-risk offenders and may 
even increase recidivism for some lower-risk 
offenders (Andrews et al., 1990; Andrews and 
Dowden, 1999; Dowden and Andrews, 1999a, 
1999b; Lipsey and Wilson, 1998; Lowenkamp 
and Latessa, 2005). Indeed, the two federal 
reentry court programs where the evalua- 
tions found promising results (C.A.R.E. and 
ACE) targeted higher-risk offenders. Studies 
with larger sample sizes and longer fol- 
low-up periods of reentry court and other 
correctional programs should provide more 
scientifically sound results to assist in future 
policy decisions. 
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BRINGING EVIDENCE-BASED PRAC- 
TICES into pretrial services is in its infancy. 
Agencies are at best beginning the process of 
considering the potential issues and assessing 
the impact of those issues on their agencies 
(National Institute of Corrections, 2008). 
The literature on bringing evidence-based 
practices into pretrial services is limited or 
non-existent (Levin, 2006; Clark & Henry, 
2003; VanNostrand & Keebler, 2007; Cadigan, 
2008). The most comprehensive attempt to 
date was just published by the state of Virginia 
and offers some promise for the future of 
evidence-based practices in pretrial services 
(VanNostrand, Rose, & Weibrecht, 2011:34). 
Most promising is that the Bureau of Justice 
Assistance agreed to fund implementation of 
the proposed evidence-based model beginning 
in 2012. In Pursuit of Legal and Evidence- 
Based Pretrial Release Recommendations 
and Supervision, by VanNostrand, Rose and 
Weibrecht (March 2011) takes a significant 
step toward filling the need for evidence- 
based work in pretrial services. 

One of the cornerstones of post-conviction 
supervision research has always been “first, 
do no harm.” Post-conviction evidence-based 
practice research has borne out that mantra 
repeatedly, showing that implementing treat- 
ment, changes, or fixes on offenders who pose 
little to no risk is fraught with failure. Is the 
same mantra true for pretrial services super- 
vision? What impact does over-supervising 
or treating low-risk defendants have on their 
outcomes? For the most part, we in pretrial 
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services have operated under the assumption 
that “it can’t hurt” to have conditions in place. 
Unfortunately, the research demonstrates 
that unnecessary alternatives to detention 
placed on low-risk federal defendants can in 
fact harm those defendants (VanNostrand 
& Keebler, 2009:10). For example, placing a 
location monitoring condition on a level-one 
defendant (the lowest-risk group) increased 
the likelihood of failure by 2.12 times, or 112 
percent over a level-one defendant who did 
not have such a condition (VanNostrand & 
Keebler, 2009:32). 

Given this evidence—admittedly, one study 
is by no means a watershed moment—the first 
step in implementing evidence-based prac- 
tices in pretrial services is to stop doing that 
which we know to be harmful: specifically, 
placing unnecessary conditions on low-risk 
defendants. This is a difficult task, as the con- 
ditions in federal pretrial services are set by 
judicial officers, generally after receiving the 
report and recommendation of a pretrial ser- 
vices officer. However, we, as officers, can and 
must at least control those recommendations 
and not recommend unnecessary conditions 
on low-risk defendants. 

The more difficult yet essential compo- 
nent is developing the extensive academic 
research and program evaluations on pre- 
trial services practices that post-conviction 
supervision currently enjoys. In addition, 
researchers need to determine what com- 
ponents from that large existing literature 
on post-conviction evidence-based practices 


pretrial services can utilize successfully. 
While the state of evidenced-based practices 
for pretrial services could be viewed as a “bare 
cupboard,” we can hope that researchers in 
this area have a real opportunity for advanc- 
ing a nascent area of the criminal justice 
literature. The number of launched initiatives 
will come together and the introduction of 
evidence-based practices to pretrial services 
will soon be commonplace and supported by 
a wealth of literature and research. 

What outcomes do we measure and what 
outcomes should be measured? Outcomes 
that have been considered relevant to pre- 
trial release are fairly standard: re-arrest, 
failure-to-appear, and technical violations by 
the defendant. Wice raises an interesting 
additional outcome that is ignored in virtu- 
ally all other studies: forfeiture rate (Wice, 
1973:66). Forfeiture rate is defined as the rate 
at which posted bonds are forfeited to the 
court primarily as a result of the defendant's 
failure-to-appear; however, forfeitures can 
also result from other violations of the con- 
ditions of release, particularly new criminal 
conduct. The State of Hawaii looked at bail 
forfeitures and concluded that 1) the vast 
majority of forfeitures were not paid and 2) 
existing policies and procedures were inef- 
fective (Hawaii, 1984:30). Given the study's 
finding of ineffective forfeiture enforcement, 
the question remains open as to whether or 
not an effective forfeiture program could 
reduce failure-to-appear and new criminal 
activity while on pretrial release. 
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The outcome measure that evidence-based 
practice needs to re-invigorate pretrial services 
is the release/detention rate. More important, 
the release/detention rate needs to be incorpo- 
rated into the more commonly used outcomes 
of failure-to-appear and re-arrest rates to 
develop a more global measure of all three 
concepts, since it is the interaction of the three 
measures that truly reflects the state or quality 
of pretrial services in any particular organiza- 
tion. Goldkamp and Gottfredson identified 
the need for a more complete measure of a 
pretrial release program that incorporates 
detention, release success, and release failure 
and described it as follows: 

Figure 10.1 [reproduced below 
as Figure 1] illustrates the simple 
measure of effective pretrial release 
that we have constructed to better 
ground performance rates in the 
context of release rates. Each col- 
umn in figure 10.1 represents 100 
percent of the defendants entering 
the criminal process in each system. 
Each column is divided into three 
parts: the lower section (black) rep- 
resents the percentage of defendants 
detained (and therefore ineligible 
to engage in misconduct), the mid- 
dle segment (white) represents the 
proportion of defendants released 
but engaging in some form of mis- 
conduct; the top portion of each 
column (gray) represents the per- 
centage of all defendants achieving 


release and not engaging in mis- 
conduct (Goldkamp & Gottfredson, 
1988:150). 

Using Figure 1 they identified successful 
pretrial as that part of the cohort released 
without pretrial misconduct. The three counties 
studied in this research clearly demonstrated 
the value of the figure, because Boston released 
virtually everyone (detaining 4 percent) but had 
significant failure (31 percent); Dade detained 
19 percent and also had significant failure (13 
percent); and Maricopa detained 45 percent, 
with 9 percent failure. Thus Dade was more 
effective (68 percent); followed by Boston with 
65 percent effective; and Maricopa last with 46 
percent effective. However, the actual success 
of the three jurisdictions was closer than the 
initial release figures would lead the reader 
to conclude at first glance. That refinement 
would seem to warrant use of the measure until 
something even more granular is developed. 

The pretrial services literature con- 
tains some excellent examples of true 
experimentation that effectively constitute 
evidence-based research. Goldkamp and 
White (1998) conducted some excellent 
experiments on pretrial services supervision, 
levels of pretrial services supervision, court 
date notification, automated call-in technol- 
ogy, and pretrial services orientation sessions 
in Philadelphia in the late 1990s. The city’s 
jail and pretrial services office had come 
under federal scrutiny due to jail overcrowd- 
ing, initially resulting in wholesale release of 
detained criminal defendants. In an effort 
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FIGURE 1. Comparison of effective pretrial release in Boston Municipal Court, Dade 
County Circuit Court, and Maricopa County Superior Court: Failure as FTA and/or rearreset. 
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to re-establish local control over the pretrial 
release system, the city hired the Crime and 
Justice Research Institute, which developed a 
response that included a number of “pretrial 
services experiments.” In addition, the mass 
release of criminal defendants prior to the 
contract with the Crime and Justice Research 
Institute created a “pre-treatment” control 
group of defendants released without pretrial 
services supervision, conditions, or any pre- 
trial services programming; this group was 
utilized as a baseline for comparison with the 
experimental groups in the research. 

Goldkamp and White used the federal 
court oversight to develop a variety of experi- 
ments and tests for various pretrial services 
programming and concepts, some of which 
were quite novel and innovative, reaching 
very useful evidence-based findings. The 
research demonstrated that pretrial services 
supervision significantly reduced re-arrest 
and failure-to-appear when compared to the 
“no supervision scenario” created by jail over- 
crowding (Goldkamp & White, 1998:76). In 
addition, that research showed that level of 
supervision (more intensive v. less rigorous) 
did not have any impact on pretrial release 
outcomes. The research also discredited the 
idea that a significant number of failures-to- 
appear are due to non-willful forgetfulness 
that could easily be corrected by telephone 
calls reminding the defendant of upcoming 
court dates (Goldkamp & White, 1998:76). 
Call-in reporting through an automated 
system was not only ineffective at reduc- 
ing failure-to-appear and re-arrest, but also 
unable to achieve any acceptable level of call- 
in compliance, with rate compliance rates of 
61 percent in the first few weeks tailing off 
steadily until week 16 when those rates fell off 
to 19 percent (Goldkamp & White, 1998:84). 
The study also found that a significant portion 
of the total pretrial failure occurs early in the 
pretrial process, with 52 percent of failures-to- 
appear occurring in the first four weeks and 
54 percent of re-arrests occurring in the first 
six weeks. Finally, the experiments concluded 
that in-person contact with a pretrial services 
officer reduced both failure-to-appear and re- 
arrest by a significant level. 

The Philadelphia pretrial services research 
conducted by Goldkamp and Gottfredson, 
Goldkamp and White, and by Goldkamp 
with a variety of other researchers over more 
than 20 years truly establishes the begin- 
nings of an evidence-based practice approach 
in pretrial services through pretrial services 
experimentation, judicial officer involvement, 


| 
| 


76 FEDERAL PROBATION 


Volume 75 Number 2 


introduction of a systemic (in this case 
guidelines-based) approach, and other work 
done in Philadelphia. The latest article on the 
Philadelphia project, by Goldkamp and Vilcica 
(2009), shrewdly brings together the majority 
of work done on the initiative and skillfully 
presents it in an evidence-based approach. 
Most important, it highlights what is missing 
from most of the other research in the area, 
and the authors conclude, quite logically, that 
their incorporation of this element, when 
no one else consistently has incorporated it, 
raises their work to something that should be 
replicated in other jurisdictions. For example, 
federal pretrial services has implemented vir- 
tually everything the other studies require, 
including a risk assessment tool; full interviews 
and reports on all defendants; untold numbers 
of alternatives to detention; quality pretrial 
services supervision; and well-trained, edu- 
cated, and compensated officers. Yet we have 
not achieved the success of the Philadelphia 
initiative. Unfortunately, even though the 
Philadelphia approach clearly seems the best, it 
has one major flaw: it has not reduced unnec- 
essary detention in any way, and that failure 
has caused more than one crisis in the opera- 
tion of pretrial services in the “city of brotherly 
love” as a result of jail overcrowding. 


TABLE 1. 


Given the amount of research that needs 
to be done to move pretrial services evidence- 
based practice forward, combined with the 
minute amount of funding and attention 
that pretrial services research has received to 
date, it seems that a new approach to assess- 
ing the value of pretrial services functions is 
warranted. For example, several years ago the 
emerging area of “reentry” caused the identi- 
fication and naming of the importance of the 
treatment and services received while offend- 
ers are incarcerated, which in turn brought 
significant attention and research dollars 
to those practices. Subsequent research has 
resulted in a variety of effective practices being 
identified, studied, and refined into even more 
effective practices. If pretrial services can be 
demonstrated to have a statistically significant 
impact on the outcomes of offenders after 
their release from prison, increased funding 
would likely follow. 

The current research is based on 79,064 
cases released on pretrial supervision between 
October 1, 2000 and September 31, 2007. The 
pretrial services data was later merged with 
post-conviction supervision data where a 
matching record, generally FBI number and/ 
or Social Security number, could be identi- 
fied. The joined file was than processed with 


TSR RPI Category—No Supervision, Supervision, Detained 


a follow-up rap sheet or criminal record check 
and subsequently analyzed. 

If pretrial services is truly important to the 
success of the criminal justice process itself, 
a defendant who is detained during the pre- 
trial period should have significantly worse 
outcomes than a defendant who is released. 
If release itself shows a positive impact, then 
the next level of analysis would be to look 
at the impact of successful pretrial release 
compared to defendants who were released 
but ultimately failed some aspect of pretrial 
release supervision. 

This first level of analysis, shown in Table 1, 
demonstrates that detained defendants are at 
least twice as likely to fail on post-conviction 
supervision as defendants who are released 
during the pretrial period. The effect carries 
for all levels of risk, except for the highest-risk 
offenders, who fail at fairly similar rates. 

Table 2 presents information on the per- 
centage of offenders that have an arrest during 
supervision based on how they terminated 
pretrial release. The pretrial release groups 
include successful, rearrested, failure-to- 
appear, and technical violation resulting in 
revocation. The “n” in the table is the number 
of failures and the percent is the failure rate 
for that group. The table presents failure rates 


Pretrial Status 


No Supervision Supervision Detained 
Risk Category N % N % N % 
Low 553/8,203 x 3,710/39,670 9 1,799/9,152 20 
Moderate 550/2,017 27 6,700/27,515 24 6,420/19,293 3 
High 315/649 49 3,734/9,039 41 8,644/17,290 50 
All 1,604/11,520 14 15,228/79,009 20 20,529/51,513 40 
TABLE 2. 
TSR PCRA Category—By Pretrial Termination Code 
Post-Conviction Pretrial Termination 
Risk Category TV NCA FTA Success 
N %o N % N % N % 
Low 103 14 43 14 16 11 1,576 6 
Low/Mod 838 29 421 31 178 28 5,365 18 
Moderate 814 52 451 53 195 50 2,942 40 
High 524 63 eay 63 140 63 1,466 32 
All 2,279 38 1,142 40 529 a7 11,349 16 
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for pretrial release in the columns and risk 
categories for the federal post-conviction risk 
tool across. The failures are identified as hav- 
ing had a new arrest while on post-conviction 
supervision. It is clear that success on pre- 
trial release leads to greater levels of success 
on post-conviction supervision. For exam- 
ple, the lowest-risk offenders fail 14 percent 
of the time when their pretrial supervision 
was revoked for technical violations; fail 14 
percent of the time when their pretrial super- 
vision was revoked for new criminal activity; 
fail 11 percent of the time when their pretrial 
supervision was revoked for failure-to-appear; 
and fail only 6 percent of the time when pre- 
trial supervision ended successfully. 

Even in the highest-risk offender category, 
offenders who successfully completed pretrial 
release were nearly twice as likely to succeed 
as pretrial defendants who were revoked: with 
63 percent of offenders whose pretrial release 
was revoked for technical violations failing; 
63 percent of offenders whose pretrial release 
was revoked for new criminal activity failing; 
63 percent of offenders whose pretrial release 
was revoked for failure-to-appear failing; and 
only 32 percent of the pretrial releasees who 
successfully ended their release failing. In fact, 
success on pretrial release appears to be more 
correlated to pretrial termination than to post- 
conviction risk category. 

This simple analysis demonstrates the tre- 
mendous capabilities that the federal system 
will enjoy as it brings pretrial services and 
post-conviction data together for the first 
time under this new infrastructure, known 
as the Decision Support System (DSS). The 
federal pretrial services and probation sys- 
tem will enjoy a distinct advantage with this 
infrastructure in leading the larger criminal 
justice community into a greater understand- 
ing of the interrelationships and impacts 
of the pretrial services system on the post- 
conviction system. 

To the larger questions of the impact of 
pretrial release on post-conviction supervi- 
sion, the first finding would seem to confirm 
a number of prior studies that show that 
defendants released pretrial do better at each 


later stage of the criminal justice process., 
The second finding that success on pretrial 
release seems to be a prelude to later suc- 
cess on post-conviction supervision does not 
seem to appear in prior literature that we 
could identify. One final word of caution 
seems warranted: while this analysis shows 
that offenders who were successful on pretrial 
release had higher rates of success on post- 
conviction supervision, we make no grand 
claims of causation. However, this initial 
result is fascinating and certainly warrants 
additional study. 
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IN 2004, THE NATIONAL Association of 
Pretrial Services Agencies (NAPSA) published 
its third edition of the Standards of Pretrial 
Release. Standard 3.7 suggests that effective 
pretrial programs and operations have mul- 
tiple characteristics, one of which is the use 
of a pretrial risk assessment tool. The past 
three decades have seen an ongoing effort in 
both pretrial programs and agencies and cor- 
rectional settings to establish the importance 
of pretrial risk assessment tools in evaluating 
the potential risk of failure to appear or more 
seriously, risk to public safety. While there are 
still pretrial recommendations based solely on 
professional judgment rather than on a valid 
pretrial risk assessment instrument, NAPSA’s 
Standard 3.7 certainly indicates the movement 
toward evidence-based practices in pretrial as 
a result of the research on best practices. In 
particular, findings from multiple studies have 
repeatedly demonstrated that actuarial risk 
assessments have a higher predictive validity 
than clinical or professional judgment alone 
(Latessa and Lovins, 2010; Agisdottir, White, 
Spengler et al., 2006; Andrews, Bonta, and 
Wormith, 2006; Grove, Zald, Lebow, Snitz and 
Nelson, 2000; Meehl, 1954). 

These studies have certainly encouraged 
the use of pretrial risk assessments, but several 
issues have arisen related to implementation 
and evaluation that have left a noted gap in 
the literature. First, some jurisdictions have 
selected tools that are either developed or 
validated on a different population without 
fully considering their target population's char- 
acteristics and what pretrial risk predictors are 


significantly correlated with pretrial failure. 
Second, jurisdictions often do not consistently 
use a structured pilot plan designed to address 
the implementation of the instrument and 
to monitor the fidelity of its administration. 
Third, pretrial agencies may not always pre- 
pare for a future evaluation of the instrument's 
predictive validity. In many cases, these con- 
cerns may be due to the high demands of the 
initial training of a new instrument, address- 
ing the cultural shift toward supporting the 
use of a pretrial risk assessment instrument 
and then meeting the overall goal of assess- 
ing the defendants and making an overall 
recommendation of pretrial supervision or 
detainment. 

Pretrial services officers work within a short 
time period to collect reliable information 
about a defendant and make recommenda- 
tions to the court concerning pretrial release 
and possible needs to be addressed dur- 
ing supervision; given such time constraints, 
the use of a predictive, objective, and reli- 
able pretrial risk assessment tool is critical 
to streamlining this process. In addition, 
NAPSAs Standard 3.4 emphasizes the impor- 
tance of incorporating information from the 
pretrial risk and needs assessment into a 
pretrial investigation report. Specifically, 
NAPSA recommends that such information 
“include factors shown to be related to the risk 
of nonappearance or of threat to the safety of 
any person or the community and to selection 
of appropriate release conditions” (NAPSA, 
2004, p. 71). Many of the factors suggested 
for inclusion in a pretrial investigation report 
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are (not surprisingly) also found on pretrial 

risk and needs assessment instruments. These 

include: 

1. Defendant's age 

2. Employment current status and history 

3. Residential history, current status, and 
community ties 

4. Criminal justice history, including if the 

defendant is on any active pretrial, proba- 

tion, or parole supervision 

Current offense severity 

6. Financial history and current financial 
resources 

7. Physical and mental health history and 
current status 

8. Substance abuse needs 

9. Previous court 
(NAPSA, 2004). 
The purpose of the current study is to 

empirically examine what risk factors are 

statistically associated with various measures 

of pretrial failure. The authors conducted a 

meta-analysis to identify these risk factors 

and to determine if some of the factors more 

commonly assumed to be associated with 

pretrial failure are actual risks. This report 

will be divided into the following sections: a 

detailed description of the methodology for 

this meta-analysis, findings, limitations, and 

last, policy implications. 


appearance history 


Methodology 


Conducting this meta-analysis took mul- 
tiple steps. First, we exhaustively reviewed 
both published and unpublished literature on 
pretrial risk assessments and risk factors of 
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pretrial. To do so, we first pulled all published 
scholarly work from an academic library's 
online abstracting resources. We searched 
for key title and subject phrases, such as 
requesting articles that included “pretrial risk 
assessment, “pretrial risk? and “failure to 
appear” in the title or in the content of 
the journal article. Next we took the ances- 
try approach, which identified additional 
research papers and articles by reviewing the 
references and citations of the articles already 
extracted. We then conducted an internet 
search using the same key title and subject 
phrase searches along with identifying and 
examining pretrial agency websites to locate 
research or studies related to pretrial risk that 
have been conducted on their populations. 
Most of these specific agency searches yielded 
studies related to the validation of a pretrial 
risk assessment tool administered by that 
jurisdiction. Finally, we directly communi- 
cated with researchers in the field who have 
conducted studies associated with the evalua- 
tion of pretrial risk assessments to identify any 
unpublished work. 

The next step for this meta-analysis was 
the development of a coding guide for review- 
ing all of the located research. This coding 
guide both helped establish eligibility for the 
study to be included in the meta-analysis and 
coded the necessary variables to conduct the 
statistical analysis and calculate the individual 
and overall effect sizes.' Variables in the cod- 
ing guide were: 

Publication information (published or 
unpublished studies, funding source, 
author affiliation) 

@ Sample demographics (sex, race, age, men- 
tal health, and sample size) 

e Pretrial risk assessment type(s) and pretrial 
predictors—which included calculating 
effect sizes for predictors as well as the 
overall assessment(s) if applicable. 

e Follow-up and recidivism - which specifies 
the length of time for the follow-up period 
and the types of recidivism measured in 
the study. 

There were four primary eligibility criteria 
for this meta-analysis. Each included study: 
(1) contained at least one outcome measure 
of recidivism for a pretrial sample, (2) pre- 
sented the statistics necessary to calculate an 
effect size, (3) was conducted in the United 
States, and (4) was published or reported after 
1960. Often, studies were excluded because 


1 


The Appendix contains a copy of the coding 
guide. Please note that variables were re-coded as 
necessary for the analysis. 


they presented only findings from binary 
multivariate logistic regression models. The 
coefficients from these models do not allow 
for calculating an effect size. For two such 
studies, we communicated directly with the 
original authors, who provided statistics that 
allowed us to calculate an effect size, thereby 
making these studies eligible for inclusion in 
the meta-analysis.* We noted methodological 
rigor of these studies in the database, since 
there were often missing data on sample 
demographics and follow-up time periods and 
minimal methodological descriptions in the 
individual studies. 

Given the number of pretrial risk assess- 
ments that were reviewed, with a variety 
of static and dynamic domains and items 
included for each specific instrument, it was 
necessary to list and calculate effect sizes for 
each possible pretrial risk factor and an effect 
size for the instrument. In addition, we created 
broad categories for all of these various items 
in the database. These included: demograph- 
ics, criminal history, personal achievement, 
residential status, substance abuse, mental 
health, and other. We calculated effect sizes 
for items and these categories based on the 
outcome measures reported in their respective 
studies. Further, we examined several outcome 
measures across the studies, in particular fail- 
ure to appear, re-arrest, new crime, and any 
failure. Any failure was used as a measure in 
several studies and was typically a combined 
recidivism measure that incorporated two or 
more outcome measures for a study. 


Analytical Design 


_ The statistical analysis for this study is straight- 


forward: we calculated random effects models. 
(We chose the random effects model, rather 
than a fixed model, since we assumed that 
these studies pulled samples from different 
populations, examined different pretrial risk 
assessment tools with unique risk factors, and 
examined a variety of outcome measures. As 
such, the random effects model suggests that 
these variations across studies could have an 
impact on the overall effect size.*) 

The effect sizes calculated for each indi- 
vidual pretrial risk predictor were Pearson's 
r correlation coefficients. Pearson’s r values 
can range from -1 to 1. A coefficient of -1 
suggests a perfect negative relationship and 


We would like to thank Marie VanNostrand, 
Ph.D. and Jim Austin, Ph.D. for their kind assis- 
tance to calculate these statistics in order for their 
studies to be included. 


3 A fixed effects model assumes that the true effect 
size would be consistent across studies. 


a coefficient of 1 suggests a perfect posi- 
tive relationship. A value of 0 indicates that 
there is no linear relationship, To interpret 
the correlation, a positive value would be 
interpreted as an increase in the first vari- 
able followed with an increase in the second 
variable. A negative relationship then would 
be interpreted as an increase in the first vari- 
able followed with a decrease in the second 
variable. Specifically, we performed these 
calculations for all individual predictors, the 
broad categories described above, the instru- 
ment, collective static or collective dynamic 
items, and then overall. As previously stated, 
these analyses are repeated for each outcome. 

In order to calculate confidence intervals 
around Pearson’ r, a z statistic was calculated.* 
This was done by converting r to z-scores, 
using the Fisher's r to z transformation.° The 
lower and upper limit confidence intervals 
around r and significance (p) values are 
reported. To interpret confidence intervals, 
the following guidelines are suggested: 

@ The smaller the range (<.10) between the 
upper and lower limit, the greater should 
be the confidence in the effect size value. 

e The larger the range (>.10) between the 
upper and lower limits, the more cau- 
tiously the effect size should be interpreted. 

¢ Similar to significance testing, the con- 
fidence interval can also suggest that the 
effect size is significantly correlated. If the 
range would include 0, then it would not 
be considered significant. 

Eligible studies’ sample sizes ranged from 
162 defendants to 202,859 defendants. Given 
this range in sample sizes, we had to address 
the potential for sampling bias, since the 
effect size from the larger studies might mask 
the effect size from studies with the smaller 
sample size. 


Findings 

There were 33 studies identified for review 
in the meta-analysis. Of these, 13 studies (k 
=13) were eligible for inclusion in the meta- 
analysis. Of these 13 studies, there were six 
pretrial risk assessment instruments with the 
necessary statistics reported to calculate an 
effect size.° These findings are presented by 


4 This z statistic was also calculated since the 


individual studies that were coded and effect sizes 
calculated for identified pretrial predictors was 
done with a meta-analysis calculator. This addresses 
issues of normality. 


> Formula for the Fisher’s r to z transformation: zr 
= (1/2)[loge(1+r) - loge(1-r)]. 


® Several of these eligible studies examined more 
than one outcome measure. 
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outcome measures. Results are provided in 
tabular format. Individual pretrial risk predic- 
tors will be presented first, then categories of 
risk, followed by static and dynamic factors, 
and the overall effect size. 


Re-arrest 


Table 1 presents the individual pretrial risk pre- 
dictors effect sizes of re-arrest. Risk items with 
the strongest significant correlations include: 
age, community supervision violation, failure 
to appear, injury to victim, instrument, jail 
incarcerations, prior conviction, prior felony, 
prior misdemeanors, prior violence, property, 
or drug and weapon. Regarding direction, the 
positive correlations for community supervi- 
sion violation, failure to appear, instrument, 
jail incarcerations, prior conviction, prior 
felony, prior misdemeanors, prior violence, 


property, or drug appear to be in the expected 
direction. For example, having one or more 
community supervision violations was signifi- 
cantly correlated with experiencing pretrial 
re-arrest. Similarly, having one or more prior 
jail incarcerations was significantly related 
to pretrial re-arrest. In contrast, the sig- 
nificant association between having a weapon 
involved in the current offense and pretrial 
re-arrest is a negative relationship. As such, 
having weapons in the current offense does 
not appear to be a significant risk of pretrial 
re-arrest. Further, if the victim is injured, 
this correlates significantly with re-arrest, but 
not in the expected direction. Likewise, age 
shows a positive relationship with pretrial 
re-arrest, but not in the expected direction 
of younger defendants experiencing pretrial 
re-arrest. Rather, this finding suggests that 


older defendants are more likely to experience 
re-arrest.’ Pretrial risk predictors that were 
not found to be significantly correlated with 
re-arrest included estimate,* family, having 
pending cases, residence length, robbery as 
a current offense, and work/employment. It 
appears that most of the significant predic- 
tors of pretrial re-arrest are static rather than 
dynamic predictors. 

Table 2 depicts the effect sizes for the six 
risk categories below. Based on these find- 
ings, criminal history, demographics, and the 
instrument overall are significantly correlated 


However, this is probably due to the different 
measures and weights for age on the various risk 
assessment instruments. 


8 Estimate is an estimate of risk that was from one 


coded study where an officer made an estimate of 
risk. 


TABLE 1. 
Effect sizes for re-arrest—Individual pretrial risk predictors 

Risk Item # Effect Sizes Correlation Lower Limit Ci Upper Limit Ci Z P Value 
Age 1 0.17 0.07 0.27 3.16 0.00 
Alcohol 3 0.08 0.05 0.11 Seen 0.00 
Communication 1 -0.07 -0.08 -0.06 -13.75 0.00 
Community 5 0.16 0.09 0.23 4.64 0.00 
supervision violation 

Custodian/Co-signer 1 0.05 0.04 0.06 8.24 0.00 
Drug 6 0.06 0.02 0.11 2.61 0.01 
Estimate 1 0.02 -0.06 0.10 0.49 0.62 
Family 2 -0.07 -0.30 0.16 -0.59 0.56 
Felony degree 1 0.11 0.10 G12 18.17 0.00 
FTA 8 0.13 0.09 0.18 5.54 0.00 
Injury to victim 1 -0.29 -0.36 -7.31 0.00 
Instrument 5 0.17 0.16 0.19 30.72 0.00 
Jail incarcerations 1 0.26 0.16 0.36 4.90 0.00 
Mental health 4 0.10 0.07 0.12 7.92 0.00 
Motivation 1 0.09 0.08 0.10 16.44 0.00 
Pending cases Z 0.04 -0.14 0.21 0.45 0.66 
Prior conviction 1 0.14 0.10 0.18 6.99 0.00 
Prior felony 2 0.14 0.09 0.18 5.69 0.00 
Prior misdemeanors 1 0.18 0.17 0.19 29:95 0.00 
Prior violence 2 0.14 0.13 0.15 24.51 0.00 
Property or drugs 1 0.16 0.08 0.24 3.95 0.00 
Residence length 2 -0.10 -0.43 0.25 -~0.55 0.59 
Residence verified 2 -0.08 -0.09 -0.07 -15.14 0.00 
Robbery 1 0.01 -0.07 0.09 0.24 0.81 
Weapon 1 0.21 -0.29 -0.13 0.00 
Work 3 0.06 -0.08 0.21 0.85 0:39 
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with pretrial re-arrest.’ Measures of personal 
achievement were not significantly related to 
pretrial re-arrest. Similar to the findings for 
individual predictors, the most strongly corre- 
lated risk categories capture static risk factors. 

Table 3 provides the effect size for static 
and dynamic predictors, for the instrument 
and the overall average effect size. Static pre- 
dictors appeared to have a stronger association 
with pretrial re-arrest than dynamic. Overall, 
the five pretrial risk assessments appeared 
to have a significant correlation with pre- 
trial re-arrest and this is close to a moderate 
strength.’ The ranges between the lower and 
upper limits of the confidence intervals were 
quite narrow. With 59 effect sizes included 
in the analysis to examine the overall average 
effect size, the correlation, while significant, is 
rather weak. 


Failure to appear 


Table 4 reports the effect sizes for the indi- 
vidual pretrial risk predictors of failure to 
appear. Risk items with the strongest sig- 
nificant correlations included age, estimate, 
failure to appear, instrument, juvenile arrests, 
prior conviction, prior jail, property or drugs, 


* Demographic measures only had one effect 


size. As such, this may not be the most reliable 
correlation. 


'° For a discussion of the strength of correlations 


for predictive accuracy and the relative strength of 


the correlation, see Rice and Harris (2005). 


and victim injury. Other than age and victim 
injury, these significant risk items were in the 
expected direction. Pretrial risk items that 
were not significantly correlated with failure 
to appear are: alcohol, communication, com- 
munity supervision violation, current felony, 
gender, residence verified, and current vio- 
lent offense. 

Table 5 presents the risk category effect 
sizes for failure to appear. All categories 
but substance abuse were found to be sig- 
nificantly correlated with failure to appear. 
Demographics, estimate and the five instru- 
ments had the strongest correlations with 
failure to appear and these three were in the 
expected direction. 

Table 6 depicts the effect sizes for the 
collective static and dynamic predictors, the 
instruments, and the overall average effect 
size. With 120 predictors examined overall for 
failure to appear, the correlation is rather weak 
at .07. The pretrial risk assessment instru- 
ments appear to have the strongest correlation 
at .19, which would be of moderate strength. 

Table 7 describes the effect sizes for the 
individual pretrial risk predictors and the out- 
come measure, new crime.'' Risk items with 
the strongest significant correlations of new 
crime were: alcohol, criminal history, failure 
to appear, juvenile arrests, prior felonies, prior 


'! With this specific outcome measure, there were 


no instruments that presented the necessary statis- 
tics to calculate an effect size. 


misdemeanors, and transportation. The rela- 
tionship between alcohol and new crime was 
negative, while the other risk items were in the 
expected direction. Risk items that were not 
significantly correlated with new crime were 
age, family, felony (current offense), residence 
length, residence verification, weapon with 
the current offense, and work. 


New Crime 


Table 8 presents the effect sizes for new 
crime. Criminal history was the only signifi- 
cant risk category for new crime and was in 
the expected direction. Demographics, other, 
residence, and substance abuse were not sig- 
nificant pretrial risk categories for new crime. 
Table 9 provides the findings for static and 
dynamic predictors and overall effect size for 
new crime. Collectively, static risk predictors 
had the strongest significant correlation, but 
this was a relatively weak correlation overall. 


Any Failure 


This last series of tables examines pretrial risk 
predictors of any failure. The strongest sig- 
nificant correlations (see Table 10) are for the 
following risk items: prior failure to appear, the 
instruments, juvenile arrests, pending cases, 
prior arrests, prior felonies, and prior misde- 
meanors. Each of these risk predictors was in 
the expected direction. Similar to earlier risk 
item tables, it would appear that these items 
are primarily static, rather than dynamic risk 


TABLE 2. 

Effect sizes for re-arrest—Risk categories 
Risk Category # Effect Sizes Correlation Lower Limit Cl Upper Limit Cl Z P value 
Criminal History 27 0.11 0.09 0.14 8.30 0.00 
Demographics 1 0.17 0.07 0.27 3.16 0.00 
Instrument 6 0.16 0.11 0.20 6.52 0.00 
Other 0.06 0.02 2.59 0.01 
Personal 3 0.06 -0.08 0.21 0.85 0.39 
Achievement 
Residence 6 -0.07 -0.11 -0.02 2.85 0.00 
Substance Abuse 9 0.08 0.05 0.10 5.03 0.00 

TABLE 3. 

Effect size for re-arrest—Static and dynamic predictors, instrument and overall effect size 
Risk Predictors # Effect Sizes Correlation Lower Limit Cl Upper Limit Cl Z P value 
Dynamic 26 0.03 0.00 0.06 2.00 0.05 
Static 28 0.11 0.09 0.14 8.52 0.00 
Instrument 5 0.17 0.16 0.19 30.72 0.00 
Overall Avg. ES 59 0.08 0.06 0.10 6.93 0.00 
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TABLE 4. 

Effect size for failure to appear—Individual pretrial risk predictors 
Risk Item # Effect Sizes Correlation Lower Limit Cl Upper Limit Cl Z P value 
Age 3 0.15 0.04 0.26 2.59 0.01 
Alcohol 4 -0.02 -0.09 9.05 -0.49 0.62 
Communication 3 0.01 -0.05 0.07 0.45 0.65 
Community 11 0.03 -0.03 0.08 0.91 0.36 
supervision violation 
Criminal history 3 0.08 0.05 0.11 4.95 0.00 
Custodian/Co-signer 2 0.04 0.03 0.05 6.71 0.00 
Drug 7 0.04 0.01 0.07 2.75 0.01 
Education 6 0.10 0.02 0.19 2.34 0.02 
Estimate 1 0.14 0.06 0.22 3.45 0.00 
Family 5 0.05 0.02 0.09 2.72 0.01 
Felony 3 -0.06 -0.28 0.17 -0.49 0.62 
FTA 13 0.20 0.17 0.22 16.96 0.00 
Gender 1 0.06 -0.03 0.15 1.24 0.21 
Instrument 5 0.21 0.12 0.29 4.55 0.00 
Juvenile arrests 1 0.13 0.12 0.14 24.51 0.00 
Mental Health 5 0.02 0.00 0.04 1:95 0.05 
Motivation 1 -0.01 -0.02 0.00 -1.82 0.07 
Pending case 8 0.11 0.02 0.20 2.39 0.02 
Prior Conviction 1 0.20 0.16 0.24 10.06 0.00 
Prior felony 3 0.11 0.06 0.16 4.33 0.00 
Prior jail 1 0.31 0.21 0.40 5.90 0.00 
Prior misdemeanors = 0.11 0.08 0.14 6.90 0.00 
Prior violence 3 0.04 0.02 0.07 335 0.00 
Property or drugs 1 0.21 0.13 0.29 Dine 0.00 
Residence length 5 0.07 0.02 0.12 2.55 0.01 
Residence verified 2 0.09 -~0.17 0.35 0.68 0.50 
Transportation 1 0.13 0.12 0.14 24.51 0.00 
Victim injury 2 0.25 -0.46 -0.01 -2.04 0.04 
Violent 2 0.02 -0.07 0.12 0.44 0.66 
Weapon 3 0.12 0.25 0.01 -1.78 0.07 
Work 10 0.06 0.01 0.11 2.21 0.03 

TABLE 5. 

Effect sizes for failure to appear—Risk categories 
Risk Category # Effect Sizes Correlation Lower Limit Cl Upper Limit Cl Z P value 
Criminal History 59 0.08 0.06 0.11 6.21 0.00 
Demographics 4 0.13 0.03 0.22 2.64 0.01 
Estimate 1 0.14 0.06 0.22 3.45 0.00 
Instrument 5 0.21 0.12 0.29 4.55 0.00 
Other 12 0.03 0.00 0.06 2.15 0.03 
Personal 16 0.08 0.03 0.12 3.02 0.00 
Achievement 
Residence 12 0.06 0.03 0.10 3.29 0.00 
Substance Abuse 11 0.02 -0.02 0.05 0.91 0.36 
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TABLE 6. 

Effect size for failure to appear—Static and dynamic predictors, instrument and overall effect size 
Risk Predictors # Effect Sizes Correlation Lower Limit Cl Upper Limit Cl Z P value - 
Dynamic 51 0.05 0.03 0.07 4.64 0.00 he 
Static 63 0.08 0.06 0.11 6.73 0.00 : 
Instrument 6 0.19 0.12 0.27 5.01 0.00 . 
Overall Avg. ES 120 0.07 0.06 0.09 9.16 0.00 

TABLE 7. 

Effect size for new crime—Individual pretrial risk predictors 
Risk Item # Effect Sizes Correlation Lower Limit Cl Upper Limit Cl Z P value 
Age 5 0.15 -0.06 0.35 1.44 0.15 
Alcohol 1 -0.16 -0.17 -0.15 -28.44 0.00 
Communication 1 -0.02 -0.03 -0.01 0.00 
Community 0.13 0.02 0.24 0.02 
supervision violation 
Criminal history 2 0.17 0.08 0.25 3.62 0.00 : 
Drug 1 0.09 0.08 0.10 15.54 0.00 : 
Education 6 0.12 . 0.00 0.24 4.95 0.05 
Family 2 0.06 -0.09 0.20 0.75 0.46 
Felony 2 0.02 -0.16 0.19 0.22 0.82 
FTA 4 0.27 0.20 0.33 oS 0.00 
Gender 1 -0.06 -0.09 -0.03 -4.61 0.00 
Juvenile arrests 1 0.21 0.20 0.22 39.96 0.00 
Mental Health 1 0.11 0.08 0.14 8.36 0.00 
Prior felony 1 0.28 0.27 0.29 53.93 0.00 : 
Prior misdemeanors 1 0.23 0.22 0.24 43.91 0.00 7 
Residence length 1 -0.01 -0.04 0.02 -0.58 0.56 
Residence verify 1 0.00 -0.01 0.01 0.00 1.00 

Transportation 1 . ‘OS 0.14 0.16 28.34 0.00 
Violence 1 0.03 0.02 0.04 6.35 0.00 
Weapon 2 0.08 -0.09 0.24 0.89 0.37 
Work 6 0.03 -0.07 0.12 0.53 O59 

TABLE 8. 

Effect sizes for new crime—Risk categories 
Risk Category # Effect Sizes Correlation Lower Limit Cl Upper Limit Cl Z P value a 
Criminal History 17 0.16 0.12 0.21 6.56 0.00 A 
Demographics ++ 0.10 -0.06 0.26 4.20 0.23 
Other 3 0.08 -0.04 0.20 1.26 0.21 
Personal 12 0.07 -0.01 O05 1.79 0.07 
Achievement 
Residence 4+ 0.03 -0.06 0.11 0.59 0.56 
Substance Abuse 2 -0.04 -0.27 0.21 -0.28 0.78 
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TABLE 9. 

Effect size for new crime—Static and dynamic predictors, and overall effect size 
Risk Predictors # Effect Sizes Correlation Lower Limit Cl Upper Limit Cl 3 P value 
Dynamic 21 0.05 0.00 0.11 1.94 0.05 
Static 21 0.15 0.11 0.20 6.41 0.00 
Overall Avg. ES 42 0.10 0.06 0.14 5.15 0.00 

TABLE 10. 

Effect size for any failure—Individual pretrial risk predictors 
Risk Item # Effect Sizes Correlation Lower Limit Cl Upper Limit Cl Z P value 
Age 2 0.07 0.04 0.09 4.35 0.00 
Alcohol 6 0.00 -0.07 0.08 0.12 0.90 
Citizenship 1 0.07 0.07 0.07 31.36 0.00 
Communication 3 0.04 -0.03 0.11 1.08 0.28 
Community 9 0.13 0.09 0.18 5.36 0.00 
supervision violation 
Custodian/Co-signer 1 0.05 0.04 0.06 8.24 0.00 
Drug 7 0.11 0.09 0.14 8.33 0.00 
Education 8 0.11 0.05 0.16 3.84 0.00 
Estimate 1 0.11 0.11 0.11 49.47 0.00 
Family 1 0.09 0.08 0.10 16.92 0.00 
Felony + 0.07 0.03 0.11 3.34 0.00 
FTA 6 0.17 0.11 0.22 5.74 0.00 
Gender 3 -0.07 -0.13 0.00 -1.96 0.05 
Instrument 5 0.16 0.14 0.19 14.37 0.00 
Juvenile arrests 1 0.19 0.18 0.20 36.05 0.00 
Medical 2 -0.03 -0.09 0.03 1.00 0.32 
Mental health 6 0.04 0.00 0.09 1.80 0.07 
Motivation 1 0.05 0.04 0.06 5c Je 0.00 
Pending cases 5 0.15 0.11 0.19 7.54 0.00 
Prior arrests 2 0.23 0.17 0.28 7.46 0.00 
Prior felony 5 0.17 0.10 0.24 4.63 0.00 
Prior misdemeanors 3 0.14 0.08 0.20 4.22 0.00 
Prior violence 1 0.11 0.10 0.12 18.17 0.00 
Race 2 0.43 -0.41 0.87 1.00 0.32 
Residence length 1 0.09 0.05 0.13 4.00 0.00 
Residence verified 5 0.05 0.00 0.09 2.11 0.03 
Transportation 3 0.08 -0.06 0.22 1.07 0.24 
Violence 1 0.00 -0.01 0.01 0.00 1.00 
Weapon 1 0.11 0.10 0.12 23.38 0.00 
Work 13 0.04 0.00 0.09 1.88 0.06 
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TABLE 11. 

Effect size for any failure—Risk categories 
Risk Category # Effect Sizes Correlation Lower Limit Cl Upper Limit Cl Z p value 
Criminal History 38 0.14 0.12 0.16 14.33 0.00 
Demographics 8 0.11 -0.18 0.39 0.76 0.45 
Instrument 6 O15 0.11 0.19 7.82 0.00 
Mental Health 0.04 0.00 0.09 1.80 0.07 
Other 0.04 -0.02 0.10 1.42 0.16 
Personal 21 0.07 0.03 0.11 3.69 0.00 
Achievement 
Residence 8 0.06 0.03 0.09 3.70 0.00 
Substance Abuse 13 0.07 0.03 0.11 3.39 0.00 

TABLE 12. 

Effect size for any failure—Static and dynamic predictors, instrument, and overall effect size 
Risk Predictors # Effect Sizes Correlation Lower Limit Cl Upper Limit Cl Z p value 
Dynamic D7 0.06 0.04 0.08 6.31 0.00 
Static 46 0.14 0.07 0.20 4.05 0.00 
Instrument 6 0:15 0.11 0.19 7.62 0.00 
Overall Avg. ES 109 0.10 0.07 0.13 5.97 0.00 


factors. Risk items that were not significant 
included: alcohol with the current offense, 
communication, medical, mental health, race, 
transportation, violence, and work. 

Table 11 displays effect sizes of the collec- 
tive risk categories for any failure. Significant 
risk categories included criminal history, the 
six risk assessment instruments, personal 
achievement, residence, and substance abuse. 
However, with the 38 effect sizes examined 
for criminal history and any failure, this 
appears to have the strongest correlation 
overall. Demographic, mental health, or other 
categories were not found to be significantly 
correlated with any failure. 

Table 12 presents the effect sizes for 
the collective static and dynamic predic- 
tors, the instruments, and the overall effect 
size. Specifically, static predictors and the 
instruments were found to be the strongest 
correlates of any failure. While all of these 
risk predictors are significant, the correlations 
overall are rather weak, although they are 
moving toward moderate strength (Rice and 
Harris, 2005). 

To briefly summarize, the purpose of this 
meta-analysis was twofold. The first goal was 
to empirically identify what risk predictors of 


pretrial are statistically associated with vari- 
ous measures of pretrial failure. The second 
intent of the study was to identify if there were 
commonly collected pretrial factors that may 
not be statistically associated with pretrial 
failure after a review of the existing research. 
The measures of pretrial failure included 


‘re-arrest, failure to appear, new crime, and 


any failure. Based on these findings, very few 
correlations were strong predictors of pretrial 
failures. Those risk items with the strongest 
correlations that were also in the expected 
direction are primarily static indicators, such 
as prior convictions, prior felonies, prior 
misdemeanors, prior failure to appear, and 
juvenile arrests. The pretrial risk assessments 
were found to be significant as well and in 
the proper direction, suggesting that increase 
in pretrial risks was significantly correlated 
with pretrial failure outcomes. To address the 
final goal for this meta-analysis, multiple risk 
predictors were examined. While there were 
insignificant predictors, several measures that 
are commonly gathered for pretrial but were 
either found to be in the wrong direction or 
were consistently not significantly associated 
with pretrial failures were: 1) residency, 2) 
injury to victim, 3) weapon, and 4) alcohol. 


Limitations 


Several limitations of this meta-analysis are 
worth noting. First, many studies were not 
included, primarily because these studies pre- 
sented only the findings from multivariate 
regression models that should not be calcu- 
lated into effect sizes. Second, some studies 
reported only presented the statistics neces- 
sary for calculating an effect size for the 
overall instrument, but not for the indi- 
vidual risk items, or vice versa. Third, it is 
likely that some studies still have not been 
located, especially for pretrial programs that 
have completed internal predictive validation 
studies within their respective departments. 
Fourth, the methodological rigor for the stud- 
ies reviewed and even for those included 
varied. This may have impacted the overall 
effect sizes that were calculated. Fifth, as 
mentioned previously, the range in individual 
study sample sizes was rather large; even with 
weights, this could potentially influence the 
overall average effect sizes. 


Policy Implications 


This study should be considered preliminary 
in examining pretrial risk predictors. Given 
the limitations previously discussed, these 
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findings must be taken within the context of 

these issues. However, within that framework, 

several policy implications can be considered. 
e Administering pretrial risk tools should 
continue. While the correlations for the 
instruments examined in this meta-anal- 
ysis were near moderate strength, these 
effect sizes were the average correlation for 
these instruments collectively (Rice and 
Harris, 2005). The range of correlations 
for the individual assessments from their 
respective studies extended from .08 to .28. 
© These assessment tools should be vali- 
dated on the population being served 
and the risk level cutoffs for these 
instruments should be normed. 

© Reliability studies should be conducted 
for agencies that have implemented a 
pretrial risk assessment instrument. Such 
studies would evaluate whether those 
administering the tool do so with fidelity 
and whether they score a given defen- 
dant similarly to other trained officers. 

@ Residency, weapons, alcohol, and victim 
injury may still be risk items for pretrial 
failure. However, future research should 
explore whether or not these factors are 
consistently related to nonappearance or 
risk to public safety. 

e For future research to continue to examine 
these goals, reports examining pretrial risk 
predictors or the validation of pretrial risk 
assessment tools should be published or 
publicly disseminated. 

@ Research on pretrial risk factors and assess- 
ment tools should report the necessary 
statistics to allow for the calculation of 
effect sizes. 
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6 unpublished data ore 
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DATE DATA GENERATED 
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2 1970s 
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3 funded, unknown source 
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Effect Size Table 
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